Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go/doc: improve headings, lists, and links #51082

Closed
rsc opened this issue Feb 8, 2022 · 150 comments
Closed

go/doc: improve headings, lists, and links #51082

rsc opened this issue Feb 8, 2022 · 150 comments

Comments

@rsc
Copy link
Contributor

rsc commented Feb 8, 2022

This proposal improves support for headings, lists, and links in Go doc comments,
while remaining backwards compatible with existing comments.
It includes a new package, go/doc/comment, exposing a parsed syntax tree for
doc comments, and it includes changes to go/printer and therefore gofmt
to format doc comments in a standard way.

For example, existing lists reformat from the display on the left to the one on the right:

URL links can be rewritten to change the display on the left to the one on the right:

And package doc comments (and others) can be rewritten to link to specific symbols:

The full design doc is at https://github.com/golang/proposal/blob/master/design/51082-godocfmt.md.

@gopherbot gopherbot added this to the Proposal milestone Feb 8, 2022
@gopherbot
Copy link

Change https://go.dev/cl/384263 mentions this issue: go/doc/comment: new package

@gopherbot
Copy link

Change https://go.dev/cl/384257 mentions this issue: all: fix various doc comment formatting nits

@gopherbot
Copy link

Change https://go.dev/cl/384267 mentions this issue: cmd/go: gofmt alldocs.go

@gopherbot
Copy link

Change https://go.dev/cl/384258 mentions this issue: all: fix TODO comment hanging indents

@gopherbot
Copy link

Change https://go.dev/cl/384265 mentions this issue: go/doc: use go/doc/comment

@gopherbot
Copy link

Change https://go.dev/cl/384266 mentions this issue: cmd/doc: use new go/doc APIs

@gopherbot
Copy link

Change https://go.dev/cl/384268 mentions this issue: all: gofmt main repo

@gopherbot
Copy link

Change https://go.dev/cl/384264 mentions this issue: go/printer: format doc comments

@gopherbot
Copy link

Change https://go.dev/cl/384259 mentions this issue: all: remove trailing blank doc comment lines

@gopherbot
Copy link

Change https://go.dev/cl/384260 mentions this issue: all: separate doc comment from //go: directives

@gopherbot
Copy link

Change https://go.dev/cl/384274 mentions this issue: internal/pkgdoc: update go/doc API usage for links

@rsc rsc added this to Incoming in Proposals (old) Feb 8, 2022
@rsc
Copy link
Contributor Author

rsc commented Feb 8, 2022

The top comment has been filled in now.
See #51082 (comment).

@jimmyfrasche
Copy link
Member

This solves 90% of my problems working with docstrings and makes the remaining 10% trivial. I am ecstatic!

I'd be perfectly happy to have this accepted as-is but I do have a couple minor thoughts:

I'd s/Text/Inline/g and then s/Plain/Text/g to align with html. That could just be familiarity but it seems clearer to me.

Allow subheadings with ##, ###, and so on. I've seen a few packages since the original discussion that could use at least one extra level of indentation for grouping subsections (I did not think to record the packages in question, sorry). This could be added later, but if so it may be prudent to accept ## x as a heading now so it doesn't change from a paragraph to a heading later. Though at that point you're most of the way to an implementation.

@rsc
Copy link
Contributor Author

rsc commented Feb 8, 2022

For now I think we should keep things simple and not increase scope past the previous discussion. There would really be no problem with having ## render as a paragraph during a later transition. Single # is rendering as a paragraph during this one.

@kortschak
Copy link
Contributor

kortschak commented Feb 9, 2022

In numbered point 5 in the goals and non-goals, the absence of in-line image support is noted for a couple of languages and support by one, but no mention of Python (a much more relevant language than for example Perl). Here it does exist in some projects https://numpy.org/doc/stable/reference/generated/numpy.fft.fft.html#numpy.fft.fft and in general, Python doc strings may be valid reStructuredText which allows embedding of images.

@rsc
Copy link
Contributor Author

rsc commented Feb 9, 2022

Python supports images in their docs, noted.
All the problems listed for images still exist though, and images remain out of scope for this change.

@ainar-g
Copy link
Contributor

ainar-g commented Feb 9, 2022

I am extremely excited about this! A couple of questions / suggestions.

  1. Currently, the pkg.go.dev renderer automatically converts text like RFC NNNN into a link to the corresponding RFC. From what I can tell, that's a feature of the website's renderer as opposed to one of godoc itself. Will that behaviour of the renderer remain intact?

  2. The point about ASCII double quotes doesn't mention if they are reformatted in the preformatted text. I think, it is worth explicitly mentioning that they won't be replaced with the Unicode ones in that position and adding test cases for that if there isn't one.

    (This may seem obvious, but anyone who ever tried editing a Shell script with the macOS TextEdit program will know that it sometimes isn't, heh.)

  3. In the numbered list section, it's currently not clear, what happens when N is above 9. I assume that it will become “␠NN.␠”?

  4. In the heading section, perhaps there should be an example of a line that begins with a space and then a #. Markdown allows a certain amount of spaces before the heading character, and if the new format doesn't allow that then it's a differrence worth mentioning explicitly, I think.

Again, really looking forward to this in any case.

@kortschak
Copy link
Contributor

Overall this looks good, but I want to restate my concern about the accessibility issues I raised with the approach to marking links.

@rsc
Copy link
Contributor Author

rsc commented Feb 9, 2022

Thanks for the questions, @ainar-g.

  1. I dont know for sure, but I think the idea is to drop the special case RFC NNNN support on pkg.go.dev and let people write links. The current support does not handle references to subsections well, for example. It was probably a mistake for pkg.go.dev to diverge this way. There is a bit more prior discussion at doc comment revisions: headings, lists, and links #48305 (comment).

  2. Doubled ASCII single quotes (`` and '') are reformatted. This is called out explicitly:

    The ASCII double-single-quote forms that have always been defined to render as “ and ” are replaced with those.

  3. If N is 10, then “␠N.␠” would be “␠10.␠”. This seems clearer than writing “␠NN.␠” - wouldn't that be “␠1010.␠”?

  4. Thanks for the suggestion. Added that as an example.

@rsc
Copy link
Contributor Author

rsc commented Feb 9, 2022

@kortschak Noted re link syntax. I don't have anything more to add beyond #48305 (comment).

@kortschak
Copy link
Contributor

It's worth noting that I did follow that up OOB with a detailed response, but did not receive a reply.

@thepudds
Copy link
Contributor

thepudds commented Feb 9, 2022

Overall, this is exciting to see.

A few quick questions about linking to headers and the relationship to the base URL.

  1. For linking to headers in the same package, would something like the following work?
See [Some Heading] for more information.

[Some Heading]: #some-heading
  1. Would that rely on knowing the default header anchor ID algorithm (or doing a copy/paste or similar)? From the API, it looks to be proposed as:
// The default anchor ID is constructed by dropping all characters
// except A-Z, a-z, 0-9, space, and tab, converting to lower case,
// splitting into space-or-tab-separated fields, and then rejoining
// the fields using hyphens. For example, if the heading text is
// “Everybody Says Don't”, the default ID is “everybody-says-dont”.
  1. Would you be able to link to headers in other packages without needing to know the base URL, such as something like:
[Another Heading]: /some/import/path#another-heading
  1. Anchor IDs removing non-ASCII letters certainly simplifies things, but if linking to something like this (one of the examples from #7349) is contemplated for the future, simply changing the anchor ID algorithm might then break other links that are mixed ASCII/non-ASCII headers?
# これは、ヘッダです

In any event, I think the ability to link to headers would be extremely useful, especially for complex packages.

@rsc
Copy link
Contributor Author

rsc commented Feb 9, 2022

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc rsc moved this from Incoming to Active in Proposals (old) Feb 9, 2022
jproberts pushed a commit to jproberts/go that referenced this issue Aug 10, 2022
The proposed (golang#51082) new go doc comment additions supports lists,
links, and doc links, but does not support links and doc links inside
lists, so implemnent this.

Fixes golang#53610

Change-Id: I4fa17d204fc9efa8f3633133e4a49e56cf1aa9bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/415174
Reviewed-by: Ben Golightly <golightly.ben@googlemail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
jproberts pushed a commit to jproberts/go that referenced this issue Aug 10, 2022
This short sentence was missing a period at the end, which caused it
to be interpreted as a heading. It also gained a '# ' prefix as part
of new gofmt formatting applied in CL 384268. This change makes it a
regular sentence as originally intended.

Updates golang#51082.

Change-Id: I100410cca21e4f91130f1f3432327bb6d66b12a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/418959
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
rinchsan pushed a commit to rinchsan/gosimports that referenced this issue Aug 14, 2022
Gofmt to update doc comments to the new formatting.

(There are so many files in x/tools I am breaking up the
gofmt'ing into multiple CLs. This is the leftovers.)

For golang/go#51082.

Change-Id: Id9d440cde9de7093d2ffe06cbaa7098993823d6b
Reviewed-on: https://go-review.googlesource.com/c/tools/+/399363
Run-TryBot: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
xhit pushed a commit to xhit/text that referenced this issue Oct 10, 2022
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: I3e3c5666d5e901f2c5303911ddb548e3dd567fce
Reviewed-on: https://go-review.googlesource.com/c/text/+/399603
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
maisem pushed a commit to tailscale/golang-x-crypto that referenced this issue Nov 15, 2022
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: I076031b6613691eefbb0f21739366e3fd2011ec9
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/399356
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
maisem pushed a commit to tailscale/golang-x-crypto that referenced this issue Nov 15, 2022
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: I076031b6613691eefbb0f21739366e3fd2011ec9
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/399356
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
WeiminShang added a commit to WeiminShang/net that referenced this issue Nov 16, 2022
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: Iae68a9cd600060577271575e893ecb23bed1e509
Reviewed-on: https://go-review.googlesource.com/c/net/+/399599
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
codegod2222 added a commit to codegod2222/oauth_go that referenced this issue Nov 25, 2022
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: Id333e34e0c5cd0bea79dcf5dd51130639597b2ee
Reviewed-on: https://go-review.googlesource.com/c/oauth2/+/399616
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
@zamicol
Copy link
Contributor

zamicol commented Dec 19, 2022

Is godoc 1.19 suppose to render the following as links?

Test line [encoding/json.Decoder] and [json.Decoder], [encoding/json] and [os.File].

The 1.19 release notes refers to https://go.dev/doc/comment#doclinks. When attempting that in godoc, it is not generating package links. (I did go install golang.org/x/tools/cmd/godoc@latest, and I'm on go version 1.19)

nikolay-turpitko pushed a commit to nikolay-turpitko/oauth2 that referenced this issue Jan 17, 2023
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: Id333e34e0c5cd0bea79dcf5dd51130639597b2ee
Reviewed-on: https://go-review.googlesource.com/c/oauth2/+/399616
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LewiGoddard pushed a commit to LewiGoddard/crypto that referenced this issue Feb 16, 2023
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: I076031b6613691eefbb0f21739366e3fd2011ec9
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/399356
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
softdev050 added a commit to softdev050/golangproject that referenced this issue Apr 5, 2023
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: Ic98f647623f234cf5d36309c6204683e151820d7
Reviewed-on: https://go-review.googlesource.com/c/example/+/399596
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
softdev050 added a commit to softdev050/Golangvuln that referenced this issue Apr 5, 2023
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: I5fd6eaa2a711d27051045aaaef45799a12777e48
Reviewed-on: https://go-review.googlesource.com/c/vuln/+/399605
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
sayjun0505 added a commit to sayjun0505/Golangvuln that referenced this issue Apr 8, 2023
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: I5fd6eaa2a711d27051045aaaef45799a12777e48
Reviewed-on: https://go-review.googlesource.com/c/vuln/+/399605
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
sayjun0505 added a commit to sayjun0505/golangproject that referenced this issue Apr 8, 2023
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: Ic98f647623f234cf5d36309c6204683e151820d7
Reviewed-on: https://go-review.googlesource.com/c/example/+/399596
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
simonswine pushed a commit to simonswine/pprof-graph that referenced this issue May 10, 2023
golang/go#51082 changed how gofmt handles some
comments, so we need to slightly reformat them to avoid `gofmt -s -d .`
errors.
CarlosRosuero pushed a commit to CarlosRosuero/typeparams that referenced this issue Jun 13, 2023
Gofmt to update doc comments to the new formatting.

(There are so many files in x/tools I am breaking up the
gofmt'ing into multiple CLs. This is the leftovers.)

For golang/go#51082.

Change-Id: Id9d440cde9de7093d2ffe06cbaa7098993823d6b
Reviewed-on: https://go-review.googlesource.com/c/tools/+/399363
Run-TryBot: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
stanislavkononiuk added a commit to stanislavkononiuk/Golangvuln that referenced this issue Jun 26, 2023
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: I5fd6eaa2a711d27051045aaaef45799a12777e48
Reviewed-on: https://go-review.googlesource.com/c/vuln/+/399605
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
BiiChris pushed a commit to BiiChris/crypto that referenced this issue Sep 15, 2023
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: I076031b6613691eefbb0f21739366e3fd2011ec9
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/399356
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
@golang golang locked and limited conversation to collaborators Dec 19, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
Development

No branches or pull requests