Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spec: use (*[4]int)(x) to convert slice x into array pointer #395

Closed
rogpeppe opened this issue Dec 8, 2009 · 118 comments
Closed

spec: use (*[4]int)(x) to convert slice x into array pointer #395

rogpeppe opened this issue Dec 8, 2009 · 118 comments
Labels
FrozenDueToAge LanguageChange NeedsFix The path to resolution is known, but the work has not been done. Proposal-Accepted
Milestone

Comments

@rogpeppe
Copy link
Contributor

rogpeppe commented Dec 8, 2009

Currently, it's possible to convert from an array or array pointer to a slice, but
there's no way of reversing 
this.

A possible syntax could be similar to the current notation for type assertions:

ArrayAssertion  = "." "[" Expression ":" Expression
"]" .

where the operands either side of the ":" are constant expressions.

One motivation for doing this is that using an array pointer allows the compiler to
range check constant 
indices at compile time.

a function like this:

func foo(a []int) int
{
    return a[0] + a[1] + a[2] + a[3];
}

could be turned into:

func foo(a []int) int
{
    b := a.[0:4];
    return b[0] + b[1] + b[2] + b[3];
}

allowing the compiler to check all the bounds once only and give compile-time errors
about out of range 
indices.
@robpike
Copy link
Contributor

robpike commented Dec 9, 2009

Comment 1:

Labels changed: added languagechange.

Status changed to Thinking.

@rsc
Copy link
Contributor

rsc commented Dec 10, 2009

Comment 2:

I think this functionality would be nice.
Personally I would rather not assume
that the compiler can subtract arbitrary
expressions (as in b := a.[0:4]) but instead
say explicitly what type I want:
b := (*[4]int)(a[0:4])
The argument against this is that we hoped
introducing x[:] would let us get rid of the
implicit conversion from *[4]int to slice.
Maybe it still does, but we allow the explicit one.
There are certainly compelling cases (mostly
in low-level things like jpeg or sha1 block
processing) where converting a slice to *[N]int
for some N would eliminate many bounds checks
for cheap.

Owner changed to r...@golang.org.

@rogpeppe
Copy link
Contributor Author

Comment 3:

> b := (*[4]int)(a[0:4])
i thought about this. the problem is that it looks like other type conversions, and
currently no 
type conversion can fail at runtime.
actually, i can't see any reason why we couldn't just use the normal type coercion
syntax:
b := a[0:4]).(*[4]int)

@rsc
Copy link
Contributor

rsc commented Dec 10, 2009

Comment 4:

Yes, that's a disadvantage of (*[4]int)(a[0:4]).
A disadvantage of a[0:4].(*[4]int) is that it takes over
syntax currently reserved for interface values.  At one
point conversion syntax and type guard syntax was 
interchangeable.  It clarified things quite a bit
to require that in x.(T), x must be an interface value
and that in T(x), the conversion must be statically
guaranteed to succeed.
Unfortunately, this particular conversion doesn't fit 
into either of those categories.
We've got enough going on that's it going to be a while
before we do anything with this.

@rogpeppe
Copy link
Contributor Author

Comment 5:

I just encountered a nice example of when this functionality might be useful. To quote
from a reddit user on why Go "needs" pointer arithmetic: 
"One well-used example is making classes as small as possible for tree nodes or linked
list nodes so you can cram as many of them into L1 cache lines as possible. This is done
by each node having a single pointer to a left sub-node, and the right sub-node being
accessed by the pointer to the left sub-node + 1. This saves the 8-bytes for the
right-node pointer. To do this you have to pre-allocate all the nodes in a vector or
array so they're laid out in memory sequentially, but it's worth it when you need it for
performance. (This also has the added benefit of the prefetchers being able to help
things along performance-wise - at least in the linked list case).''
You can *almost* do this in Go with 
   type node struct {
      value int
      children *[2]node
   }
except that there's no way of getting a *[2]node from the underlying slice.

@gopherbot
Copy link

Comment 6 by nmessenger:

If neither syntax is ideal, perhaps a new unslice builtin?
    ary, ok := unslice([n]T, slc)
Though should ary have type [n]T or *[n]T? If n is large and the unslice fails, a large
zeroed array might not be ideal. Anything wrong with this that I'm not seeing? Well,
besides it being another new builtin.

@gopherbot
Copy link

Comment 7 by robpike:

This is a big deal because ary would not have static type.

@rsc
Copy link
Contributor

rsc commented May 17, 2011

Comment 8:

If we were going to do it - which is far from even up for debate - I think
the syntax x.(*[10]int) works well (x has type []int).  You can't .(T) a slice
type right now, so it would not be overloading anything, and like an
interface type assertion it can fail or be checked at run time.
You can even think of it as []int containing a *[10]int the same way
an interface value holds a concrete type, and you're extracting the
underlying array pointer.
That said, I don't think this is important enough to worry about now.
There is enough else going on.

@rsc
Copy link
Contributor

rsc commented Dec 9, 2011

Comment 9:

Labels changed: added priority-later.

@mdempsky
Copy link
Member

Comment 11:

Since this is something that's been bugging me lately too, I thought I'd add a few
random thoughts I had that don't seem to have been mentioned:
Allowing conversions from slices to array-pointers means pointers can now refer to
partially overlapping objects.  I don't believe that's currently possible in the
language.  Slices can already overlap though, so it's not a big change overall.
Like #8 says, []T is sort-of an interface type for *[N]T, so type assertions are
arguably suitable syntax.  Except that cap(x.(*[N]T)) might give a different value than
cap(x), which isn't true for other interfaces/implementor-type relations.  Seems like an
open question whether this inconsistency is worth accepting into the language, and
really since there's already a way to convert a *[N]T into a []T, just the ability to
turn a []T into a *[N]T is the relevant missing feature.
It would be nice if an expression like x[e1:e2] could actually have a static type of
*[e2-e1]T (assuming e1 and e2 are constant expressions), then you could write something
like *dst[16:24] = *src[136:144] and the compiler can verify that the array bounds match
up.  Unfortunately, the expression can't actually be x[e1:e2] since existing code might
rely on cap(x[e1:e2]) == cap(x)-e1, and that would be a backwards incompatible change. 
The x.[e1:e2] syntax suggested originally would solve this issue.
If you want a range like x.[n:n+4], instead of requiring the language to recognize this
pattern somehow, x[n:].[:4] is equivalent and has static indices at the expense of
clunkier notation.  A short-hand notation like x.[n:+4] might be nice to indicate that 4
is a length not an end position, but not strictly necessary and complicates the
language.  (Also +4 here is technically ambiguous here whether it's length 4 or end
position "+4", so again some new notation would be necessary.)

@gopherbot
Copy link

Comment 12 by peter.waller:

I just want to note that we have a use case for this at go-gl. More information:
go-gl/gl#111
The underlying OpenGL API only accepts the equivalent of, e.g, a *[4]float32, so it is
nice to have this in the type system on our side. OTOH, a consumer of this API might be
holding a []float32 they want to pass to us. So it would be great to find a solution to
this, as the current solutions are a bit messy or require the use of unsafe.

@rsc
Copy link
Contributor

rsc commented Nov 27, 2013

Comment 13:

Labels changed: added go1.3maybe.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2013

Comment 14:

Labels changed: added release-none, removed go1.3maybe.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2013

Comment 15:

Labels changed: added repo-main.

@ncw
Copy link
Contributor

ncw commented Jan 20, 2014

Comment 16:

An alternative which occurred to me was in a function which only indexes a slice with
constants values (eg the JPEG routines or unrolled FFTs which are what I'm working on)
and the slice doesn't change the compiler could bounds check the slice just once at the
start of the function with min(constants) and max(constants).
This would achieve the removal of the bounds checking without a language change.  It
wouldn't allow the compiler to do range checking at compile time though.

@gopherbot
Copy link

Comment 17 by matthieu.riou:

Another use case is to be able to use a small slice of known length as a map key. The Go
API uses slices heavily even though the same length is expected. Being able to get the
underlying array would allow usage as map keys.
Two examples I've run into recently are hashes (SHA-256) and IP addresses (as 16 byte
slices). It seems rather wasteful to have to copy them or transform them to strings to
have to use them as map keys.

@nerdatmath
Copy link
Contributor

FWIW, something similar to unslice() above can be implemented with reflect and unsafe. Despite being implemented in terms of unsafe, I believe unslice itself is safe. I don't know whether it violates any assumptions made by the GC, however.

http://play.golang.org/p/DixtgwxXUH

@daviddengcn
Copy link

Actually I believe the compiler could easily be smart enough to make a single range check for statement like this:

return a[0] + a[1] + a[2] + a[3]

@chalonga
Copy link

Is there a good way to do this currently?

For example if one function gives me a slice as output and I need to use that output in another function that wants a fixed array as input. What is the best way to coerce the slice into an array that is the current size of the slice and containing it's current members?

@ianlancetaylor
Copy link
Contributor

ianlancetaylor commented Apr 14, 2015

Currently there is no safe way to convert from a slice type to an array type (that is the point of this issue).

You can do it using unsafe by writing code like

(*[10]byte)(unsafe.Pointer(&b[0]))

@betawaffle
Copy link

@bcmills: Thanks for the suggestion. Opened #45545.

gopherbot pushed a commit that referenced this issue Apr 20, 2021
Implementation follows in subsequent changes.

Updates #395

Change-Id: Ic97ee822805e4c236fdd9d224e776cb2ae62c817
Reviewed-on: https://go-review.googlesource.com/c/go/+/216424
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
gopherbot pushed a commit that referenced this issue Apr 21, 2021
Panic if the slice is too short.

Updates #395

Change-Id: I90f4bff2da5d8f3148ba06d2482084f32b25c29a
Reviewed-on: https://go-review.googlesource.com/c/go/+/301650
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
gopherbot pushed a commit that referenced this issue Apr 21, 2021
These match the changes to cmd/compile/internal/types2 in CL 301650.

Updates #395

Change-Id: I1e85b6355c8c8fdba0996c26a2505c65fab908d6
Reviewed-on: https://go-review.googlesource.com/c/go/+/301651
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/312070 mentions this issue: cmd/compile: allow export/import OSLICE2ARRPTR

gopherbot pushed a commit that referenced this issue Apr 21, 2021
Updates #395
Fixes #45665

Change-Id: Iaf053c0439a573e9193d40942fbdb22ac3b4d3bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/312070
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/315169 mentions this issue: cmd/compile/internal/types2: slice-to-array-pointer conversion requires go1.17

gopherbot pushed a commit that referenced this issue Apr 29, 2021
…es go1.17

Add missing version check. Even though this is a new types2 error
we separate between the compiler and the types2 error message: we
have the compiler error message to match the compiler style, and
we have a types2-specific error message to match the types2 style
for these kinds of errors (for now).

Eventually we need to decide which style we like better and clean
this up.

Follow-up on https://golang.org/cl/301650.

Updates #395.

Change-Id: I5b779f345994c66b1f4a4db466466f98b7d3c491
Reviewed-on: https://go-review.googlesource.com/c/go/+/315169
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
@gopherbot
Copy link

Change https://golang.org/cl/325549 mentions this issue: doc/go1.17: document slice to array pointer conversion

@gopherbot
Copy link

Change https://golang.org/cl/327649 mentions this issue: cmd/compile: allow ir.OSLICE2ARRPTR in mayCall

gopherbot pushed a commit that referenced this issue Jun 13, 2021
CL 301650 adds conversion from slice to array ptr. The conversion
expression may appear as argument to a function call, so it will be
tested by mayCall. But ir.OSLICE2ARRPTR  op is not handled by mayCall,
causes the compiler crashes.

Updates #395
Fixes #46720

Change-Id: I39e1b3e38e224a31f3dec46dbbdc855ff3b2c6a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/327649
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/334669 mentions this issue: reflect: add Value.CanConvert

gopherbot pushed a commit that referenced this issue Jul 21, 2021
For #395
For #46746

Change-Id: I4bfc094cf1cecd27ce48e31f92384cf470f371a6
Reviewed-on: https://go-review.googlesource.com/c/go/+/334669
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
@gopherbot
Copy link

Change https://golang.org/cl/336890 mentions this issue: doc: clarify non-nil zero length slice to array pointer conversion

gopherbot pushed a commit that referenced this issue Jul 26, 2021
There is an example for nil slice already, so adding example for non-nil
zero length slice, too, clarifying to the reader that the result is also
non-nil and different from nil slice case.

Updates #395

Change-Id: I019db1b1a1c0c621161ecaaacab5a4d888764b1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/336890
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/338630 mentions this issue: compiler, runtime: allow slice to array pointer conversion

gopherbot pushed a commit to golang/gofrontend that referenced this issue Aug 2, 2021
Panic if the slice is too short.

For golang/go#395

Change-Id: I184f87d0207dcee4be6b36ae446b84e9583b356d
Reviewed-on: https://go-review.googlesource.com/c/gofrontend/+/338630
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
nstester pushed a commit to nstester/gcc that referenced this issue Aug 2, 2021
@gopherbot
Copy link

Change https://golang.org/cl/339329 mentions this issue: compiler: check slice to pointer-to-array conversion element type

gopherbot pushed a commit to golang/gofrontend that referenced this issue Aug 3, 2021
When checking a slice to pointer-to-array conversion, I forgot to
verify that the elements types are identical.

For golang/go#395

Change-Id: I533ac52c0b390af96fce78a8c468ae9d8ad79da9
Reviewed-on: https://go-review.googlesource.com/c/gofrontend/+/339329
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
nstester pushed a commit to nstester/gcc that referenced this issue Aug 3, 2021
When checking a slice to pointer-to-array conversion, I forgot to
verify that the elements types are identical.

For golang/go#395

Reviewed-on: https://go-review.googlesource.com/c/gofrontend/+/339329
@danieljl
Copy link

the length of the slice must equal the length of the pointed-to array

Why == instead of >=? The conversation above has pretty consistently discussed >=.

Maybe it's a bit too late to comment since the feature is already landed on Go 1.17, but I just want to point out that the >= behavior might lead to a hidden bug due to a different behavior expected by many programmers, i.e. they expect the conversion will success only if the lengths are equal. It violates the principle of least surprise.

The == behavior is also more explicit, which is usually preferred by the Go community, than the >= behavior: If the original slice's length is larger than the destination array's length, the slice needs to be first resized to the array's size.

@golang golang locked and limited conversation to collaborators Aug 24, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge LanguageChange NeedsFix The path to resolution is known, but the work has not been done. Proposal-Accepted
Projects
No open projects
Development

No branches or pull requests