Re: Why doesn't foreach return a value

On Sat, 2 Feb 2008 19:47:13 -0800 (PST),
"tom.rmadilo" <tom.rmadilo@xxxxxxxxx> wrote:

On Feb 1, 5:56 pm, Darren New <d...@xxxxxxxxxx> wrote:
tom.rmadilo wrote:
Maybe it is more accurate to
say that Tcl evaluation does not return to the script if an
exceptional condition is returned by any command found in that
script.
No, it isn't.
The script is destroyed, evaluation stops and control is returned
to the enclosing context.
No, it isn't.
I guess what you guys should do is submit a massive request to update
the Tcl documentation, because the distinctions I have made appear to
be present in every case:
Some commands return a meaningful value:
gets, if, regexp, etc.

They return a NON-EMPTY value.


Some commands return a meaningless value:
foreach, while, for

They return an EMPTY value.

If they returned an uninitialised string pointer, which might be
pointing into some random spot in program memory, or worse, maybe into
some random stop in memory that'll fault when you try to read it, then
THAT would be a meaningless value. But empty vs. non-empty is still
perfectly valid, and hence potentially useful.


As a case in point, [proc] returns an empty value, which by your
definition, is meaningless. Yet this type of construct works, and I use
it myself on occasion as a way of wrapping an initialisation script
being executed at global scope so that it doesn't leave useless global
or namespaced variables hanging around (another use was demonstrated in
my posts about my [keyed] convenience construct):

[proc "" {args} {
foreach arg $args {
... do something with $arg ...
}
}] arg1 arg2 arg3 ,,,

Notice that it's creating a proc with an empty (but still meaningful)
name, and then executing it as a quick "lets process this stuff within
a [proc] context" one-shot command thing. The [proc] returns an empty
string, which is re-interpreted (being substituted into the first word
of the command) as the name of a command, that we happen to have just
finished creating. Note that the empty result value is very different
from, say, a C void return value that is a special case entity.

Further, [foreach] and co. often occur right at the end of a proc, for
which reason I was a little hesitant to suggest letting the inner loops
return value carry through as the default return value. Some people
rely on [foreach] (and friends) returning an empty value, to also make
their proc return an empty value. Although, moving into 8.5 where we
have a decent [return] command, I think that's a practise we need to
try and get away from.

But, the short of it is that these values are only "meaningless" if you
choose to think of them that way. And that's entirely your problem.


Some commands don't return a value (they don't return to (but abort)
the script where they were invoked, and maybe cause higher up scripts
to abort as well, all without returning):
break, continue, return, error

But that's a fallasy which we've been pounding on for ages. Two of them
return real values, and the other two return empty ones, along with a
flow-control signal. If you invoke them through [catch], you'll get
that return value and flow-control signal just like normal. In
standard code the flow-control signal is handled automatically, to
guide the how and why of that return value (which may itself be
considered irrelevant and generally ignored, and so is set to be
empty mostly symbolically), unless caught and handled differently, but
it's still being returned. And it's still as valid a value as any
other. You could initialise variables with the return value of
[break], by way of a statement like this:
catch {break} initialise-me
So how can you possibly say it doesn't return a value, or does return a
meaningless value. It DOES return an EMPTY value. And how can you say
that's not returning to the script where it was invoked? That's
returning exactly to where it was invoked.

What you need to be thinking, instead, is that the script from where it
was invoked it automatically re-throwing the return value, in the case
where it's not one that's expected (which for regular code is exactly
TCL_OK).

foreach arg $args {
if { $arg eq "foo" } {
break
}
}

Here you have three levels of code.

[foreach] is invoking {if {$arg eq "foo"} break}. It does this by
handing off the script to an evaluator function.

The evaluation breaks the script into commands, and executes each one
in turn. After each command, it checks the return code, and continues
on to the next one only if it's TCL_OK. If it's anything else (in this
case, potentially TCL_BREAK), then it handles it differently.

[if] does the same. It invokes the expression evaluator to handle the
expression, and the script evaluator to handle the body (in this case
just the command [break]). In both cases it checks the return code
first, to see if it has to do anything special, and only then bothers
to look at the actual return value. And in both cases the evaluators
themselves check return codes of anything they hand off to be performed
by something else.

So between the [foreach] command and the [break] command, something
like this happens... (I'm ignoring TCL_ERROR, for example)

- a script evaluator running the [foreach] command
- the [foreach] command is running through a list of values (or two)
- another script evaluator is spawned by [forwach] to handle the body
- the [if] command is being executed by that last script evaluator
- it has spawned an expression evaluator to test the conditional
- the expression evaluator returned true, and TCL_OK for flow control
- [if], happy with that, then spawned another script evaluator
- that script evaluator executes the [break] command
- [break] returns TCL_BREAK, leaving the default blank result value
- the evaluator (from if) wants only TCL_OK, so re-throws the TCL_BREAK
- [if] simply passes back what it was given by its script evaluator
- the next evaluator (from foreach) does exactly the same as the last
- [foreach] now receives the TCL_BREAK with an empty result value.
- it knows how to handle TCL_BREAK; stop and return TCL_OK
- the outermost script evaluator receives TCL_OK, and continues on

As a point of interest, [catch {break} var] looks basically like this:

- a script evaluator running the [catch] command
- [catch] spawns another script evaluator to run the command
- the new script evaluator executes the [break] command
- [break] returns TCL_BREAK, leaving the default blank result value
- the evaluator (from if) wants only TCL_OK, so re-throws the TCL_BREAK
- [catch] handles everything;
- stashes the result value (empty) in "var" (was given here)
- would have set the options dictionary if it were asked to
- converts the TCL_BREAK code into an integer value
- returns the integer value with the return code TCL_OK
- the starting script evaluator continues on

It's a little more complicated than that, since this [catch] is
probably a value being tested or assigned to a variable. But the
routine is basically the same;

- split the command into distinct words
- if a word includes a nested command
- invoke a script evaluator to evaluate that command
- check for TCL_OK, and re-throw anything we don't like
- if it's TCL_OK, substitute the result value back in our place

So at every point, both inside and outside of script evaluators, the
return code is being checked, and generally if it's anything other then
TCL_OK (0), the script evaluator stops what it's doing and throws it
back, ditching the rest of the script fragment. Otherwise, it just
moves onto the next command (if there is one), returning the last
result value with a TCL_OK return code if that was the last one.

And then whatever invoked the script evaluator itself will check the
return code, which will be TCL_OK unless it was re-throwing something
it didn't like, and either acts on it itself (be it TCL_OK, or
TCL_BREAK in a [foreach]), or just re-throws it, often straight back to
another script evaluator.


Now, having written all that, if I hear once more you saying that
[break] and [continue] don't return anything, there's a VERY good
chance that I'll be killfile'ing you. ;)


Also, the documentation is as confused as I am about return codes,
saying that they are used to control flow, and have nothing to do with
the return value.

No. That is absolutely correct. Return codes, being the ones literally
returned from functions at the C level, do indeed have nothing to do
with the return value, which is stored independently within the
interpreter.


But I'm sorry to report that the commands which don't return a value,
according to the documentation, all use a similar phrase 'returns a
code ... which causes' something to happen.

Conceptually, the return code and the response value are considered as
a single return value unit, for the simple reason that you never have
one without the other.

It's impossible not to return a code. There is exactly one type of
function in C which returns, literally, nothing. And it returns void.
In the TCL code, ALL script-level commands, and the evaluators used to
invoke them, return an integer return code, which in all but a couple
very exceptional cases, MUST be checked and if not handled, re-thrown.
The very top-most level will then in turn translate anything other than
a simple TCL_OK return code into an error and quit. This translation
may or may not make use of the interpreters result value, which like
the return code, ALWAYS exists, though as DKF pointed out, it gets set
to an empty value before any tcl-level command is evaluated (presumably
from within the script evaluator prior to dispatching each command).


The documentation is also annoyingly consistent about what script is
aborted and where control is directed to continue.

Well, I hope I've cleared some of that up for you. TCL_OK generally
continues on to the next command in a script fragment, anything else
will cause a disruption to the normal flow, generally causing whatever
it is to stop where it is, and return the code (and value) it presently
has back upwards to the previous stack frame.

And I can't imagine any exception to this rule not being clearly
stated, for those exceptions are generally the specific purpose of the
command involved.


Also, those bytecode substitutions, like totally skip over code when
replacing break and continue. Probably the bytecode stuff is wrong,
you can't just skip over all that important returning stuff.

Darren answered the rest of this beautifully. Read it very, very
carefully.


To recap the key points, for anyone still confused;

1) There is ALWAYS both a return CODE, and a return VALUE.

2) The return value, unless changed, will be the empty string.

3) Script level is a virtual concept. It doesn't exist. It is, in
fact, the side-effect of a script evaluator launching commands on your
behalf. This means that things happen between and around TCL commands
which can appear magical when viewed entirely from within TCL scripting.

4) Your script will never see a return code other than TCL_OK (or its
associate value), except in specific cases like [catch], because the
glue I just mentioned in point 3 that makes script commands work, will
have handled it already. This does not mean that it doesn't exist.

5) With most non-TCL_OK return codes, the value will be thrown away and
ignored unless you specifically catch it, again, most often using the
[catch] command. That same glue does however handle TCL_ERROR, which I
have left out of this message for simplicity sake. Where exactly did
you think those great long error messages came from?!?

6) Everything in TCL is a command, everything conforms to these rules,
and there is neither magic nor exceptions. All commands return, unless
TCL has been crashed, because return is the method used to unwind the
stack frame, and the stack frame must be unwound for things to continue.

7) The documentation attempts to extract from this a simpler reality in
which there is magic to handle the parts that you shouldn't need to.
But in participating in this discussion, you attempted to reach beyond
the reality the documentation paints, to the much finer-grained reality
described in the six points above, where there is no magic, just a lot
of rules and recursive machinery that constructs the seemingly complex
magic out of pain-staking consistency and simple repetitive actions.

As the saying goes; if you can't stand the heat, get out of the
fire. Just sit by, obey the documentation, and warm your feet at a
distance. :)


Fredderic
.

Relevant Pages