Re: Assembler error + large data

Daniel Carrera <daniel@xxxxxxxxx> wrote:

I have a problem and I'm not sure if it is specific to GFortran or
generic. My program needs to read a very large data file and I am
getting a strange assembler error. It's clearly to do with the size of
my data. I have written a toy program that illustrates the error:

integer, parameter :: nrows = 233280000
integer, parameter :: rp = real64
real(rp), dimension(nrows) :: longitude, latitude, height, area

When I compile I get this:

gfortran -o test.o -c --std=f2008 -fimplicit-none -Wall -fbounds-check
/tmp/ccKqApNC.s: Assembler messages:
/tmp/ccKqApNC.s:31: Error: value of 000000014db5b000 too large for field
of 4 bytes at 0000000000000061

It means that the array is too big for your compiler.

Specifically, the compiler works by generating an assembly language
program and sending that to an assembler, which then finds that
it is too big.

You need a 64 bit version of the compiler, with matching assembler.

The other possibility is to figure out how to do it without
reading in the whole array at once. It might be necessary for
your problem, but many people naturally first think of reading
in a whole file, no matter how big, instead of considering how
to process it one record at a time.

If I understand it right, each array is too big, and you have
three of them. For a smaller array, it might compile, but then
fail at link time or load time, when the whole program is found
to be too big.

How much memory do you have on your computer? 16GB is affordable
these days, though not so many machines will hold that much.

-- glen

Relevant Pages

  • Re: How difficult is this to implement?
    ... perhaps also try unrolling the loop for a few thousand ... Mind you, practically speaking, to out-optimise a compiler *will* be ... come with linking my assembler code piece to C that Rod has mentionned. ... For me it is essential that the migration through the array is as fast ...
  • Re: How difficult is this to implement?
    ... relatively clueless about assembler. ... it's very difficult to out optimize a C compiler. ... as the first or last calculation (Can you tell I've done this before? ... if you have an array, you can also setup a reverse array so you can ...
  • Re: History of byte addressing
    ... separate user and OS versions of the compiler and assembler - and that=20 ... and an OS version of the assembler. ... There is a C compiler now. ... It declared all of the C data memory as a large array of byte, and pointers as an index in the array. ...
  • Re: What micros do you actually hate to work with?
    ... with less than tiny asm ... hassle to work with 32 bit longs in assembler for an 8 bit uP, ... Microchip's C compiler tools and their assembly under MPLAB. ... is that other programmers for C are easier to find. ...
  • Re: The Promise of Forth
    ... Declare variables, declare locals, let the compiler do the work ... Forth could be considered the assembler for a simplified virtual ... If you use locals then you have to initialise them each time. ... there are likely to be libraries that could be profitably used within ...