Re: The Linear Address Space

From: flekso (taurus_at_email.hinet.hr)
Date: 04/27/04 

Date: Tue, 27 Apr 2004 16:36:43 +0000 (UTC)

I wrote the reply in maximized window, so i guess it would be best viewed as
such.

> I have been working through the IA-32 Software Developer's Manuals. I have
> read pretty much all of it in some detail (skipping mostly just tables and
> lists that can be referenced as needed). Still there is something very
basic
> in my understanding that is lacking.

Don't skip the tables :).
Better yet get a book Intel Microprocessors 8086 .. Pentium 4
(http://users1.ee.net/brey/p8.htm). Much more comprehensive than intel's
stuff. I plan to use the first book to get more understanding of the topic
then use Intel's stuff as a reference/update.

> There is a Physical Address Space of 4GBytes (2^32 bytes) which basically
> identifies all of the binary numbers that can be used as addresses for
> assembly language programming.

That's virtual address space. Physical is the amount your DRAM sticks add up
to.

> With the 36-bit address extensions it is possible to increase this to
> 64Gbytes.
> The problem is that in a computer there is Hard Drive storage, RAM storage
> and a heap of other devices and the like that all require byte addresses.
> Say for example you had a 180GByte HDD in your system, even with the 36
bit
> address extension mechanisms which bring the Physical Address Space up to
> 64GBytes, there are still just not enough Logical Addresses to reference
the
> whole of the addresses on a system.

I'm not sure about the whole ata standard on i/o, but i'm pretty sure that
it doesn't require direct map of every hdd byte into virtual space.
Maybe a small amount of ram is occupied by memory mapped i/o registers and
cache/buffers, and then you write your ata commands to these registers and
get the results in cache (not sure about all this though).
Also, standard size for disk i/o packet(sector) is 512 bytes at once, so, a
36 bit pointer would allow you to access quite a lot of 180GB drives at
once... (2^36 * 512 bytes).

>
> Also with Hard Drive partitions and with every program having to be put in
a
> particular spot on the Hard Drive, it is a very rare thing that one would
> write a program that is to be put at address zero.
> >From all this I have come to think that the Linear Address Space is
simply
> an abstraction or a virtualization of the true state of all of the
addresses
> within a computer so what I need to know is how are these abstractions
that
> are used in a program "mapped" to the real addresses of the computer?
> How does this work?
>
> Also, I think that paging has something to do with RAM management and the
> "checking in" and "checking out" of data to and from RAM but frankly I can
> neither confirm nor deny this and hence I just cannot figure out what is
> actually going on with paging.
> Am I on the right track with paging? - Please correct!
>
> Might the mapping to real addresses have something to do with the Page
> Directory Base Register (PDBR) which is Control Register 3 (CR3)?
>

Few key terms associated with virtual/paged memory:

Memory is 'formatted into 4KB blocks named page(s) and whole thing revolves
around that.

Page directory - its *physical* base address is located in register CR3.
Contains 1024 real pointers to page tables.

Page tables - each page table contains 1024 real pointers to pages. (real =
physical address, virtual address is just a pack of three indexes to the
real tables/pages).

Pages - in a 32-bit addressing system, page represents 4KB of physical
address
space.

Dir
----Table0
        ----Page0
        ----Page1
        ----Page2
        ----Page3
        ...
----Table1
        ----Page0
        ----Page1
        ----Page2
        ----Page3
        ...
...

Each table covers an area of 1024 * 4096 bytes (1024 pages in a table * size
of a page) = 4MB. Multiply that by the number of tables in a directory and
you
get 4GB (1024 tables * 4MB covered by each table).

Linear(virtual) 32-bit address is composed of three indexes (left 10 bits
are directory index, middle 10 bits are table index, and last 12 bits are
offset in the selected page).
2^10 = 1024 (max. number of tables in a directory, also max. number of pages
in a table).
2^12 = 4096 (size of the page, ideal for offset).

So virtual address 00000000h would have a meaning (it could point anywhere,
even to physical 00000000h :) ):

10 bits 10 bits 12 bits ..total of 32
bits
[0000000000][0000000000][000000000000]
dir_index 0 tbl_index 0 offset 0

How cpu accesses ram in pseudo C:

unsigned long *phyAct;
unsigned long *phyPD;
unsigned long *phyPT;
unsigned long *phyP;

phyPD = cr3 ; // get page directory base address

phyPT = *( phyPD + ( dir_index * 4 ) ) ; // add dir_index * 4 (entries in
directory and tables are 4-byte) to phyPD and dereference to get the table
address

phyP = *( phyPT + ( tbl_index * 4 ) ) ; // add tbl_index * 4 to phyPT and
dereference to get the page

phyAct = *( phyP + offset ) ; // add offset to phyP to get the actual
address pointed to by this whole mess

Earlier i stated that the directory/tables contain 4-byte pointers to
tables/pages but that's not accurate, only the leftmost 20 bits are used as
a pointer, but since pages/tables are 4KB, for 32-bit addressing scheme that
can access 4GB max that's no problem ( 2^20 = 1MB, times 4KB pages = 4GB).
Remaining bits are used for page properties (accessed,privileged...)

So pagedir[dir_index] yields a real address of a table,
pagetbl[tbl_index] yields a real address of a page, and
page[offset] gets us to the much desired memory location.

All this is done by cpu alone, no assembly required.

When i first read this stuff, i started making all kinds of
assumptions(performance wise), declaring war to all things paged, but was
swiftly quieted down by the more knowledgeable on this group.
The thing is all ia-32 processors since 486 have some form of translation
lookaside buffer (TLB) implemented in them that effectively eliminates(most
of the time) this monkey business, only the first access to the page goes
through the procedure
described above and once translation is over the real address is stored in
the buffer.

I haven't even entered segment protection because, as they've told me it's
not on the cool side of things anymore... And myself buried deep behind the
ms-dos trenches last couple of weeks trying to create a small partition
conversion utility i'm beginning to see the point.
Windows/linux uses only one(or is it two) segments for all its stuff so who
am i to complain.

Try these links for more (comprehensive) info:
http://www.uni-tuebingen.de/zdv/projekte/linux/books/khg/section2_7_9.html

http://www7.informatik.uni-erlangen.de/~msdoerfe/embedded/386html/c05.htm

Relevant Pages

  • Re: Whip a duo-core PC into submission...?
    ... used are the Pointer type, which on Win32 contains the 32 bit ... the memory symmetrical in halves. ... 2GB process running on each core (ignoring OS and other ... overheads) in which case I naively assume that 4 GB of RAM ...
    (comp.dsp)
  • Re: Declaring static pointers.
    ... a pointer to my house might be: ... However, in the current context, we refer to an address in memory, ... ROM and RAM might exist in completely different ...
    (comp.lang.c)
  • Re: Memory Limitations?
    ... I still don't see how system RAM and video RAM access are related. ... //pointer is a pointer type which is 32bits wide ... //video_display is the video card's memory which will display on ...
    (microsoft.public.windowsxp.customize)
  • Re: Declaring static pointers.
    ... a pointer to my house might be: ... has asked about two distinct kinds of memory with different ... ROM and RAM might exist in completely different ... The large ROM region uses 32-bit addresses while ...
    (comp.lang.c)
  • RE: Windows Server 2003 Enterprise Edition installation
    ... if I have say 10 GIG of RAM but I only have a pagefile of 4096 Meg? ... Makes me personaly wonder if virtualization has flaws ... |> less swapping and guests should run from memory.. ...
    (microsoft.public.windows.server.setup)