Re: NRE committed: PLEASE TEST

Fredderic wrote:
On Fri, 22 Aug 2008 09:52:18 -0300, miguel <msofer@xxxxxxxxxxxx> wrote:

Fredderic wrote:
Personally, I think it would make more sense if [yield] would
function regardless of whether the proc was evaluated via
[coroutine] or not, rather than throwing an error. You could still
use [coroutine] to create simultaneous invocations, and unlike a
[coroutine] invocation, the original proc it would essentially
"reset" when it completes. Basically acting exactly as though it
was storing internal state in a private global variable between
runs.
Not sure about that ... when [yield] fails to accomplish what it is
designed to do, I think it is better for it to error out rather than
do something else.

How is it not doing what it's designed to do? If it fails because of
an impure stack, then by all means error out. But [yield] would be a
handy way to allow a [proc] to do some initialisation on first run.

Oh - now I understood what you mean. This would make procs a bit heavier than coroutines (see below). As currently implemented, it would also prevent the second and later calls to take more than one argument.

IOW, your proposal may look like "let us change [yield], but ir really is "let us change [proc]".



The only issue I'm aware of, is how they would mix. [yield] would end
up yielding the current [coroutine] instead of creating a new one. (I
presume in your implementation, you can [yield] from any TCL call
depth.) It would need a [yield -local] switch (or its converse) to
function the way I would like to see it, I guess. I'd like to know
which most people think is the sanest default behaviour if both were
available; yield back to the inner-most [coroutine] and error if there
isn't one, or yield the current proc only which is more what the term
co-routine implies to me. (In either case there would be an option to
perform the other.)

I think you are talking about stackful versus non-stackful coroutines, in the terminology of http://www.inf.puc-rio.br/~roberto/docs/MCC15-04.pdf. The experimental coroutines in HEAD are stackful, what the term implies to you seems to be non-stackful.

Something that would at least let me fudge what I would like to see, is
the ability for [coroutine] to push the current proc out of the way and
bring it back later. So you'd end up with something like this:

proc something {args} {
coroutine something {{abc def} {
{ ... the real body goes here ... }
}} {*}$args
}

I'll certainly be adding a -coroutine switch to my standard [proc]
wrapper, I think, though it's going to be ugly and flakey without that
kind of capability.

I presume there'll be an [info] sub-command to tell us whether we're in
a [coroutine] or not...?

What is required for [info] is still an open question. Also a way to query if a given command is a coroutine or not might be useful. Or if [yield] will succeed. (as opposed to catching it). I'm waiting to see what the users need.

Failing that, I think [coroutine] should work exclusively on
lambda's and not [proc]s at all, to highlight the fact that you
seemingly can't really use them in interchanged contexts; a proc
with [yield]s won't work too wekk when invoked directly rather than
via construction by [coroutine], and a proc without [yield]s
doesn't make much sense to invoke through [coroutine]. So you'd
take a lambda style function-in-a-variable, and use [coroutine] to
create a command out of it, which would then be used instead of
invoking the lambda via apply.
Conceptually (and very much in code too), what
coroutine foo cmd arg1 arg2 arg3
does is:
1. create a new execution environment

Could you elaborate on this part, please? What is different in this
new execution environment compared to the regular one, and how heavy is
this new environment. Would it be feasible to do this as part of the
regular proc invocation process?

By my understanding this basically entails starting a fresh TCL stack,
and pre-loading it with a small hook to regain control later on, within
which you evaluate the subject code.

I suppose my question, is whether there's anything within the TCL stack
that will stop you from basically taking the current TCL call level
with you to this new environment. As I understood it (which may well
be utterly wrong here ;) ), the TCL stack portions were relocatable, it
was (and still is) the C chunks that got in the way, which is what the
NRE eval deals with by allowing the C stack to unwind before performing
the eval.

In the current implementation of CallFrames the previous levels at creation time cannot be preserved, and that is why the coroutine runs at level #0. Preserving the complete CallFrame stack is not impossible, but it is not a minor undertaking as a lot of infrastructure has to change. It may happen for 8.7 or 9.0 if interesting use cases warrant it, but that's not within a reasonable horizon for 8.6

The Tcl stack portions are not relocatable, no: there are pointers to them being saved (eg by [upvar] used in child CallFrames, or stuff that is allocated using TclStacklloc). The change you propose would imply that the Tcl stack goes, that CallFrames are allocated independently (and refcounted). Something like that may yet happen during the 8.6 cycle, I have some ideas that would make the cost of separate allocs disappear. But it would be purely an internal change, just an enabler for other things that may lie in the future.

2. create a new command foo that resumes running this execEnv
3. swap in the new execEnv as the current execution environment,
insuring that the caller execEnv is swapped back in when this
one exits (return or yield)
4. run (in the new env) uplevel #0 [list cmd arg1 arg2 arg3]
5. if the execEnv returned, delete it together with the associated
command foo

Steps 2-5 sound like standard library-based task-swapping mechanics.


Yield just checks that the current environment is not the main one,
ie, that we are executing within an execEnv that can be suspended. It
does not know about procs, lambdas nor anything else.

It was the fact that [yield] will work from any level that had slipped
my attention. Still, having [coroutine] take a lambda instead of a
proc would make it nestable within a proc, as in the example up top,
and I do still think it makes a whole lot more sense, as I said, with
[yield] not working from the main environment, even though [yield] can
still occur within other procs.

I wonder whether it would be sane to add an [info lambda] command that
will compile and return a proc as a lambda expression...?

I'm afraid I did not understand much of this. You say "having [coroutine] take a lambda instead of a proc" [it takes neither, it takes a command to run in the new env] would make it nestable within a proc [I do not understand what you mean here]".


The other thing it checks is that the C stack is currently at the
same level it was when the execEnv was last resumed. This means for
instance that you can't [*] yield from within an [eval]ed or
[source]d script.

I presume these two can be fixed, though...? Which leaves just
old-style non-NRE-aware extensions.

I hope to be fixing these two during the alpha/beta cycle. I am really worried about [eval], not so much about [source].

Some other things do preempt yielding, traces come to mind and may yet be fixed (haven't even explored what it would take).

.

Relevant Pages

  • Re: NRE committed: PLEASE TEST
    ... to displace an existing proc or command until it completes. ... that [yield] would only effect the proc it was invoked from). ... If the original command has been replaced by a default coroutine, ...
    (comp.lang.tcl)
  • Re: NRE committed: PLEASE TEST
    ... be a proc, and hence you could pass in arguments and have them update ... so you give an argument to the coroutine and it pops out as the ... that a design decision, or a practical limitation? ... foreach arg [info args $proc] { ...
    (comp.lang.tcl)
  • Re: NRE committed: PLEASE TEST
    ... My wish here would be served by the command being able to ... displace an existing proc or command until it completes. ... A limitation of this approach is that you couldn't have 2 or more simultaneous coroutine instances based on the same command. ... If you think about how the mechanism works, a coroutine suspends from a yield, and is later resumed from that place. ...
    (comp.lang.tcl)
  • Re: NRE committed: PLEASE TEST
    ... resume only takes a single argument, whereas the original proc ... Creation of a coroutine takes inital arguments - fair enough - and resuming a coroutine also takes an argument. ... From the Tcl script level it is indeed a command, and it does take an optional argument. ...
    (comp.lang.tcl)
  • Re: NRE committed: PLEASE TEST
    ... function regardless of whether the proc was evaluated via ... use [coroutine] to create simultaneous invocations, ... But [yield] would be a ... swap in the new execEnv as the current execution environment, ...
    (comp.lang.tcl)