Re: Tcl 8.4 can kill Windows XP

From: David Gravereaux (davygrvy_at_pobox.com)
Date: 03/16/05 

Date: Tue, 15 Mar 2005 18:55:18 -0800

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Roy Terry wrote:
> "David Gravereaux" <davygrvy@pobox.com> wrote in message
> news:l6qdncSZveNax6rfRVn-pA@speakeasy.net...
>
> Roy Terry wrote:
>
>>"David Gravereaux" <davygrvy@pobox.com> wrote in message
>>news:_ZqdnXkAu_N3marfRVn-pg@speakeasy.net...
>
>>Roy Terry wrote:
>
>
>>>"Thomas Schnurrenberger" <tsbg@swissonline.ch> wrote in message
>>>news:d0qgho$hnd$1@news.hispeed.ch...
>
>
>>>>Roy Terry wrote:
>
>
>>>>>Can anyone suggest a good and quick tool to
>>>>>log i/o calls from the wish/tclsh process into the OS?
>>>>>
>>>>>I have procexp.exe from sysinternals.com but that
>>>>>doesn't trace calls.
>>>>>
>>>>>Thanks,
>>>>>roy
>>>>>
>
>>>>Have you tried the Filemon utility from Sysinternals?
>
>>>Good. Yes that shows Tcl just doing the
>>>obvious create/write/close as near as I can tell.
>
>>I can smell that one coming.. and it smells like a windows OS bug.
>
>>I did the same amount of tracing for a winsock bug once and finally
>>determined it was the notification mechanism of WSAAsyncSelect getting
>>overrun. But that was for FD_ACCEPTs getting lost by winsock coming in,
>>your's appears to be file creates getting lost writing to disk.
>
>>We could try rewriting the file channel driver for overlapped mode ;)
>>How does the notification work in the file channel driver? I've never
>>looked at it.
>
>
>>>I don't know if file events play into this as the only events
>>>I'm using are "after" events.

Whoops. somehow thought [fileevent] could do notifications.. It
doesn't really make sense, does it? You can't wait on a file becoming
readable in the same way you do for a socket? I guess there isn't
really a mirror for 'tail' on windows is there?

In concept, should there be a way to tell.. Guess you read all the way
to EOF, close and wait for more FindNextChangeNotification(s) to fire
then pick-up from the last file pointer spot, repeat. That would do it,
I think. But then you aren't waiting on the handle, you'd be waiting on
the file itself which doesn't fit the Tcl innards model all that well.

> But I'm with you that it's probably
>>>something in the OS. After 300 files/sec for about 1hr 30 minutes
>>>the OS hangs! - Still investigating.
>>>Roy
>
> I'll have a look and try to rough a replacement in overlapped mode.
>
>> Would that approach be a win in efficiency and throughput?

- From what I can gather from the docs, more context switching is required
for many/short files and the benefit is for large/few files. But.. we
are already using buffered I/O provided by the generic layer, so there's
probably not much of a benefit moving from synchronous to asynchronous
ReadFile/WriteFile except that writes can go really fast. In
synchronous, it only syncs to the fact that the file system consumed the
buffer, not that the data was written to disk. It'd make a big
difference if we were to write a gig of data in one call to WriteFile,
but we don't; the generic layer chops it up as per -buffersize.

>> UPDATE: installed XP Pro SP1a on unused disk and booted
>> that version. Ran the stress test (300 files/sec, 5000/500 bytes sizes and
>> it was still running fine after 4.5 hours, 4,560,000 files. Hmm. Now
>> re-running
>> on SP2 and assuming it will hang as usual. If so I'll be loading all MS
>> updates/patches
>> and re-try. Right now it looks like they put their foot in it on SP2.

I just looked at trying overlapped and it isn't going to be that easy
for me. It isn't going to be quick as it'll require different code for
NT and 9x systems and I'll have to track the file pointer manually. The
9x code will require a thread pool due to the 64 handle limit for
WaitForMultipleEvents, but the NT code can just use a single thread
managing the IO completion port..

Dang.. So close to what I want to do for that IOCPSOCK thing when I
place it in the core. I'd better just do that first ;)

- --
David Gravereaux <davygrvy@pobox.com>
[species:human; planet:earth,milkyway(western spiral arm),alpha sector]

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (MingW32)

iD8DBQFCN6AUlZadkQh/RmERAhTXAJ4x1hQZKlH1W0VNJE59NJieWualkgCg9Bhz
OUHXCHgPwhhORFpDdPah6lE=
=lXol
-----END PGP SIGNATURE-----

Relevant Pages

  • Re: Why RosAsm Breaks on a large number of symbols
    ... > Windows message loop is a good starting point. ... But of course this is not a genuine bug report, ... > preservation convention for Windows programing, ... You need to preserve register in callbacks - that's the only _rule_ about ...
    (alt.lang.asm)
  • Re: bad experience with Suse 9.1 on Inspiron 8200
    ... > it is an M$ bug, such technicalities are irrelevant to the end user. ... :-) Windows has ... work as long as they work, trying to install a nitrous ... Linux are at least a *little* bit curious about how their computer works ...
    (alt.os.linux.suse)
  • I think I have been hijacked.
    ... I am running windows xp on my Compaq Presario and Toshiba laptop, ... An internet connection appears to have been added through a USB. ... R - Registry, StartPage/SearchPage changes ... Fixed crashing bug on certain Win2000 and WinXP systems at O23 listing ...
    (microsoft.public.windowsxp.security_admin)
  • Re: Given Up on Linux1
    ... Once I had done enough work and investigated the existing bug reports on ... > dealt with thousands of computers, you should be lucky I'm even ... newbie to Linux. ... But there had not been a windows problem in 12 years I ...
    (alt.os.linux)
  • Re: WMP 11 Recently Added function...
    ... I could quote Zach's statement about how terribly expensive a single bug fix is and how the right thing to do when a bug is identified is not always to fix it. ... Or I could tell you that he has often said that the only way to report a bug in Windows Media Player is to call product support. ... experience of wmp not showing recently added files and the id tags problem with dpoweramp, ...
    (microsoft.public.windowsmedia.player)