Confused about 'capturing'-behavior of ( ... ) e.g. "immutable lists" vs. [ ... ] e.g. "arrays"

Question

I'm somewhat confused about the behavior of the following Raku program,

my @list;

my $v = 4;

@list.push: [ 1, $v ];
@list.push: ( 2, $v );


$v *= 2;

@list.push: [ 3, $v ];
@list.push: ( 4, $v );


$v *= 2;

for @list -> $item
{
    say "'{$item}', type '{$item.WHAT.raku}'"
}

The output of this program (with Rakudo™ v2023.11) is:

'1 4', type 'Array'
'2 16', type 'List'
'3 8', type 'Array'
'4 16', type 'List'

I actually expected the following output:

'1 4', type 'Array'
'2 4', type 'List'
'3 8', type 'Array'
'4 8', type 'List'

With this in mind, consider the following (similar) raku REPL interaction:

[0] > my @list;
[]

[1] > my $v = 4;
4

[2] > @list.push: [ 1, $v ];
[[1 4]]

[3] > @list.push: ( 2, $v );
[[1 4] (2 4)]

[4] > @list[1][0] = 3;
Cannot modify an immutable List ((2 4))
  in block <unit> (...)

[4] > @list[1][1] = 3;
3

[5] > $v
3

The error caused by evaluation 4 seems to confirm my idea that comma-separated expressions become "immutable lists" (which is what I want/expect - something akin to Python's tuple-type), but it seems that they 'capture' the scalar container?

Is this really intentional? Because I'm under the impression that behavior like this (which may be useful) should require some kind of explicit "syntax" (like \( ... ))?

If it is intentional, what do I need to understand e.g. 'internalize' in order to explain this behavior?

Thanks for your clarification!

Edit: SO question 77628295 'captures' my issue as well. one of Raiph's answers to this question refers to this answer which describes it well: "($bar,...) is a list containing $bar, not $bar's value".

About my confusion (and perhaps the confusion of others as well): I've picked up the habit of using Python's tuples as "immutable lists" with the knowledge that such structures are more compact (e.g. memory efficient) than mutable (and growable/shrinkable) arrays while doing so also expresses some intent: "this is generated data, do not mutate". In my personal case, I was generating "AST-like structures" in a Raku program. This concept cannot be mapped to Raku, however. With Elizabeth's workaround (provided for question 77628295) in mind, I've now prefixed integer values with 0+ and string values with ""~ inside this program where I want to generate such lists.

This behavior therefore does seem to be intentional. The explanation makes sense to me but i'm still quite surprised. I guess I stepped into a (to me) newly discovered pitfall.

Answer 1

I eagerly await @raiph's answer since he usually explains with a depth and clarity that I don't.

In the meantime, here's my answer:

• I recommend you start by reading the page on raku Lists, sequences, and arrays
• I recommend you take a look at the Array typegraph and go to the docs pages to quickly compare the methods provided by Arrays and Lists
• Also consider the Literal constructors and note the way that the , comma operator makes Lists and the contrast with the [] operator and the special working of Array assignment
• One more relevant point is that a raku List (and a Seq) is by default lazy, whereas a raku Array is by default eager

Now we have some common ground that a raku Array inherits from and is a more functional beast than a raku List.

A key statement in the docs is During assignment, each value will be typechecked to ensure it is a permitted type for the Array. Any Scalar will be stripped from each value and a new Scalar will be wrapped around it.

[Actually I think that this statement should be tweaked to 'When it is loaded, each value...' since afaict it applies to push, append, unshift and so on]

So, given that we already know that a raku Scalar such as $v is an itemized container that may hold a value, then the mysterious behaviour you describe is a clear consequence of the way that raku chooses to assemble these building blocks.

Why is it this crazy way, you ask? Here's my take in the form of a counter example:

• Imagine if you apply the Array loading approach to a List - so 'Any Scalar will be stripped from each value and a new Scalar will be wrapped around it.'
• Oops you just iterated all the elements ... so this List can't be lazy after all

[0] > my @a = [^10]            #Array
[0 1 2 3 4 5 6 7 8 9]
[1] > my @b := (^10)           #List (lazy by default)
^10
[2] > my @b := (^10).eager     #List (eager)
(0 1 2 3 4 5 6 7 8 9)

Sooo, to resolve this, the raku language design does not even provide a mechanism to review and maybe extract and copy over Scalar values when Lists are assembled. The Scalar is simply bound as that container in that position when it is encountered. And the value (which is only ever one err value) can be changed by assignment wherever it is in scope.

Now, one could argue that Python Tuples have it right and raku Lists have it wrong, but I believe that (i) there is a natural logic to having Lists do the simple, lazy thing and Arrays do the rich and eager thing and (ii) these two classes between them cover all the bases that coders need. So I would argue that the raku way is a better way to do it.

This divergence from common languages does need us to explain and document that raku is a new approach on listy things and perhaps we need more on this in the docs. But when you grok what is going on under the hood (ie not much), then actually it kinda makes sense that Lists (ie the simple variant) do not bother to copy over values but just bind the Scalar container.