Re: Collecting like-labelled sublists of a list

On Jul 30, 6:45 am, Volkan YAZICI <volkan.yaz...@xxxxxxxxx> wrote:
On Jul 30, 1:36 am, Cortez <relativef...@xxxxxxxxxxxxx> wrote:



My original version -

; cpu time (non-gc) 0 msec user, 0 msec system
; cpu time (gc) 0 msec user, 0 msec system
; cpu time (total) 0 msec user, 0 msec system
; real time 0 msec
; space allocation:
; 76 cons cells, 0 other bytes, 0 static bytes

Alberto -

; cpu time (non-gc) 16 msec user, 0 msec system
; cpu time (gc) 0 msec user, 0 msec system
; cpu time (total) 16 msec user, 0 msec system
; real time 15 msec
; space allocation:
; 691 cons cells, 944 other bytes, 0 static bytes

Volkan -

; cpu time (non-gc) 0 msec user, 0 msec system
; cpu time (gc) 0 msec user, 0 msec system
; cpu time (total) 0 msec user, 0 msec system
; real time 0 msec
; space allocation:
; 327 cons cells, 944 other bytes, 0 static bytes

I don't think your original version will keep its speed stable --
because of (member index indices) searches -- as input grows. Maybe
you should also tell us more about the average # of labels.

Regards.

Hi Volkan,

Well a 1MB SDIF file might contain about 3000 partials, so that's 3000
labels. That's about the size of file I typically work with. But the
situation is complicated by the fact that within the file itself these
partials are distributed over a set of time-ordered matrices, which
have to be traversed in order to extract the data. Each matrix
contains information on every partial located at that point (each
matrix also has a header). The information itself (frequency,
amplitude, phase, etc - although I'm only interested in frequency at
the moment) is typically in the form of single or double floats. For
example, here are three matrices (which I model as arrays) containing
data extracted from a single clarinet note. The label is in the left-
most column, followed by frequency, amplitude and other data which I
won't elaborate on here (relating to time-offset and noise content.
The time of the matrix itself is contained in the matrix header). So
you can see how the partials (0 to 6 in this example) are distributed
across the matrices -

#2A((0.0d0 1207.8944079486291d0 3.0572778826667964d-5
0.8507826862315859d0
0.9633105769790011d0 8.735621320690784d-4)
(1.0d0 2799.767798804896d0 2.9758011552006322d-5
3.45542871985445d0
0.9999134899562377d0 7.948914515372436d-4)
(2.0d0 556.677039133396d0 1.7342238369471644d-5
3.5441699661680883d0 0.0d0
0.0013469001214222072d0))
#2A((0.0d0 1203.4944446791235d0 2.7734811951076312d-5
3.7002160311387104d0
0.8199124994997562d0 4.790926227113963d-4)
(1.0d0 2660.5866387929855d0 2.5368686049752616d-5
3.5907080654740575d0
0.9996524472597946d0 3.0160728537549295d-4)
(2.0d0 606.6191777974843d0 6.739337893175857d-6
2.3996345665170553d0 0.0d0
0.0011529762653688502d0)
(3.0d0 2324.4893423205413d0 4.0769660018709746d-5
3.41151701297919d0
0.9974054018288645d0 3.8066262505557277d-4)
(4.0d0 5335.909767981719d0 3.5703443068263075d-5
3.111386950381825d0
0.9988603457041186d0 4.2724354268293147d-4))
#2A((0.0d0 1201.692599868909d0 3.140013470961943d-5
2.6591665451516757d0
0.6600595944811739d0 7.04389350218825d-4)
(3.0d0 2345.9740698897076d0 2.501488942774825d-5
3.3084674908933707d0
0.9955756776543997d0 3.5830861408908156d-4)
(4.0d0 5335.896889017415d0 3.63802335700116d-5 6.002182744750073d0
0.9999136282420076d0 1.5051639945999097d-4)
(5.0d0 1963.8199112992056d0 2.1833696069379764d-5
3.411731878197788d0
0.9837995802145886d0 0.0012953992561072873d0)
(6.0d0 3466.815966572441d0 4.645427928279146d-5
0.4173643804422096d0
0.9949535597520733d0 5.296405453024079d-4))...

The project I'm working on is a personal Lisp library for reading and
writing SDIF files (it's also a general exercise for me in writing an
application). I already have a working version, but I want to make it
much more efficient. It's not that slow (and in SBCL is actually very
fast), but now I'm trying to write a simple graphical interface, in
which the data is visualized. I'm experimenting with McClim and ACL.
Both get quite slow for large files. At the moment I'm just reading in
all the data into a slot in one of the graphical objects, and then
passing it to the various drawing functions I use. What I want,
ideally, is to do the drawing as I'm parsing the file.

I model the internal structure of an SDIF file through various
classes, instances of which are used to hold the data (SDIF is a
binary format). My current method of extracting partials is to read
all of the matrices into arrays (see above) then iterate over these
for each partial, starting from 0, until I've found them all. This is
horribly inefficient, of course. So the original function I posted was
a model (admittedly simplistic) for the kind of thing I want to do
instead, which is to extract the partials as I'm actually reading in
the bytes.

BTW, more info on SDIF can be found at: http://recherche.ircam.fr/equipes/analyse-synthese/sdif/
.