Skip to main content

Reading partially downloaded gzip with an offset [Resolved]

Let's say that there is one huge db.sql.gz of size 100GB available and the server supports range requests.

So instead of downloading the entire file, I downloaded y bytes(let's say 1024bytes) with an offset of x bytes(let's say 1000bytes) like the following.

curl -r 1000-2024

With the above command I was able to download the partial content of the gzipped file, now my question is how can I read that partial content?

I tried gunzip -c db.sql.gz | dd ibs=1024 skip=0 count=1 > o.sql but this gives an error

gzip: dbrange.sql.gz: not in gzip format

The error is acceptable since I guess at the top of the file may be there are header blocks which describes encoding.

I noticed that if I'm downloading the file without an offset, I'm able to read the file using gunzip and piping.

curl -r 0-2024

Question Credit: bravokeyl
Question Reference
Asked September 17, 2019
Tags: gzip, streams
Posted Under: Unix Linux
2 Answers

Your Answer