Re: Reasons for a buffer or RAM



pensul wrote:

> On page 82 of "Programming from the Ground Up" we're in the midst of a program
> that will convert all the lowercase letters from a file into uppercase letters.
> Specifically the buffer section that assignes memory for the buffer looks like
> this:
>
> .section .bss
> #Buffer - this is where the data is loaded into
> # from the data file and written from
> # into the output file. This should
> # never exceed 16,000 for various
> # reasons.

I'd like a little more discussion on the "various reasons",
actually... I had a situation where I wanted to read a
CPM-86 disk from Linux, and was trying to decide the size of
the buffer to use. The maximum size of a CPM-86 partition is
8M, so I figured some even fraction of this would be good...
Then I decided to try for an 8M buffer, and read the whole
thing at once - not really expecting it to work (won't work
in dos!!!). But it worked fine... That's not to say it's
"good practice", but it worked. It would be better, I think,
to have allocated such a big buffer dynamically, so that it
could be "free"ed when I was done with it. But it worked...

> .equ BUFFER_SIZE, 500
> .lcomm BUFFER_DATA, BUFFER_SIZE
>
> Previously it was explained that we didn't know how big the first line would
> be, and that we needed to put it into a buffer.

True. Of course, even if we knew the size, we'd still have
to put it in a buffer. We *could* use just a one-byte
buffer, and process the file a byte at a time, but this
would be quite slow (not that you'd notice it on a modern
machine...). Better to call the OS as infrequently as we
can, and do as much as possible with each call.

> Consequently, we might
> unexpectedly run out of space. However, doesn't the file itself already exist
> somewhere in memory or disk space? It must needs be so, because if our programs
> cannot access memory directy, but only thru the kernel, how much more will a
> random input from a user

You really shouldn't *allow* "random input from a user". You
should always limit the *amount* that the user enters to the
size of the buffer you've got available. And then you should
check to make sure they've entered valid data. You can't
trust users to enter a number, if asked for a number, for
example. Pesky things, users. Pity we can't figure out a way
to eliminate 'em :)

> be incapable of using up all the memory without the
> kernel knowing about it? So my question is: if the kernel already knows the
> size of the file, why can't it pass this information on to the program, so
> that it can set its memory requirements?

Well, you can. You can "stat" the file (or seek to the end
of it) to determine the size, and then "malloc" a buffer big
enough - better yet, mmap the thing... but not on page 82 :)

Jonathan's just trying to start out simple, as "From The
Ground Up" would suggest...

Best,
Frank
.



Relevant Pages

  • CONFIG_PACKET_MMAP revisited
    ... I've been looking into faster ways to do packet captures and I stumbled on ... In that discussion Jamie Lokier suggested having a memory buffer that's ... shared between user and kernel space and having the NIC do DMA transfers ...
    (Linux-Kernel)
  • Re: contigmalloc() and mmap()
    ... there seems no big differences between the kernel ... > to the card on another node, it will be DMAed to memory too. ... The buffer is mmaped to user process space, ... > mmap driver's buffer (allocated by contigmalloc()) and is killed, ...
    (freebsd-hackers)
  • Re: [2.4] heavy-load under swap space shortage
    ... > This file controls the operation of the bdflush kernel ... > -to a clean buffer, which can just be forgotten about). ... > -value can lead to memory shortage when bdflush isn't woken ... > Kswapd is the kernel swapout daemon. ...
    (Linux-Kernel)
  • Re: contigmalloc() and mmap()
    ... there seems no big differences between the kernel ... > to the card on another node, it will be DMAed to memory too. ... The buffer is mmaped to user process space, ... > mmap driver's buffer (allocated by contigmalloc()) and is killed, ...
    (freebsd-hackers)
  • Reasons for a buffer or RAM
    ... that will convert all the lowercase letters from a file into uppercase letters. ... Specifically the buffer section that assignes memory for the buffer looks like ... kernel knowing about it? ...
    (alt.lang.asm)