Re: Sliding window


"filia&sofia" <in_tyrannos@xxxxxxxxxxx> wrote in message
news:6fe1ca80-5232-4d44-9503-7a62a20b03cc@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

I want only the bits that exist in the file originally. The length of
the chunks should be ceil(n/8) bytes. Also, the bits that do not hold
any information are located at the "Most Significant Bit" end. In
short, the read information (in bits) is represented in bytes.

For example, if the file has: "0110 0100 1111 1101 0011 0101 0101 1100
0000" and n=9.
Then, 1st chunk is (without padding) "0110 0100 1". 2nd is "111 1101
00". 3rd is "11 0101 010" and 4th is "1 1100 0000".
Now, we need two bytes to represent nine bits. We pad all the chunks
with zeroes, e.g. 1st becomes "0000000 011001001" etc.

Will the alignment on output be the same as in the input?

The output file is same as the input file: the algorithm has to remove
padded zeroes and join chunks in the way that preserves information in
overlapping bytes. At this point, the algorithm should produce
identical files in a extremely fast manner.

Hopefully, this additional information helps.

Actually, it's made things more confusing! I've looked at your earlier post
and from that it seemed you need some sort of combined move+bitshift.

I assume your N in that post (buffer byte size) can be bigger than a machine
word? And the M bits at a time you're taking from that buffer can also be
bigger than a machine word?

Best to post some C code here, so that assuming it works as you want, you
might be able to get help with optimising.

--
Bart


.

Relevant Pages

  • Re: Expat problems
    ... > buffer is nul-terminated. ... Not a standard header. ... was read in by "chunks" instead of the whole file into memory. ... I say read in the data in chunks. ...
    (comp.lang.c)
  • Re: How to split a compressed file programmatically?
    ... value is there in converting that to a string and then back to a long? ... And why allocate a new buffer for each chunk you want to write, ... void splitFile(string path, string path_parts, int size_part) ... just read chunks of the original file until you can't ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: How to split a compressed file programmatically?
    ... Personally, I would forget about the calculation altogether and just write a loop that keeps writing bytes in chunks as large as you want or however many bytes you have remaining, whichever is less, until you have no more bytes to write. ... The "size_part" variable is already a long; what possible value is there in converting that to a string and then back to a long? ... And why allocate a new buffer for each chunk you want to write, and why does that buffer have to be the length of the original file, and given that you're allocating a new buffer each time, why read the data anywhere other than the beginning of the buffer? ... void splitFile(string path, string path_parts, int size_part) ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: serving a file to a client --- background fcopy read fileevent event puts socket cha
    ... probably not -size (although having -size pre-load the input buffer ... you can't even observe the progress of the [fcopy] from outside (ie. ... 4096 byte chunks. ... The division of functionality I mentioned should have negligible impact ...
    (comp.lang.tcl)
  • Re: [ANSI C] cross-platform fgets()
    ... you can't read from disk in per-line chunks. ... > character by character, watching for new-lines and carriage returns. ... Ideally you should read a chunk of the file into a buffer - a ring ... typedef struct filebuffer ...
    (comp.programming)