WIP: Json documentation generation by daspecster · Pull Request #1775 · googleapis/google-cloud-python · GitHub

scripts/generate_json_docs.py is now generating parsable JSON for the gcloud-common site generator.
As well as building a types.json file.

Several things are still in progress...

Extracting code examples is not working consistently.
The table of contents needs to be updated. (just switched to listing all classes)
Class linking should be added. i.e. class:gcloud.datastore._generated.query_pb2.Query
Source Code links are broken for base class docs
Not yet sure how to handle __init__.py's
"Getting Started" and "Guide" type pages need to be ported.(done for what we have but should add more).
Running generate_json_docs.py needs to be added to the build process and documented better.
Find all directives that are not filtered and filter/handle them (for now anyway).
Possibly add auto-href generation for link in text for easier navigation out site docs.
Figure out how to handle snippets. This should probably be in a different PR after this launches
Bring in Code snippets Add ability to pull snippets when generating JSON documentation #1951
Trim back TOC.
Switch to live gh-pages committing

We found a Contributor License Agreement for you (the sender of this pull request) and all commit authors, but as best as we can tell these commits were authored by someone else. If that's the case, please add them to this pull request and have them confirm that they're okay with these commits being contributed to Google. If we're mistaken and you did author these commits, just reply here to confirm.

@daspecster From https://api.github.com/repos/GoogleCloudPlatform/gcloud-python/pulls/1775/commits it appears some of the commits have @jgeewax and @callmehiphop emails in them (which is why the CLA bot is angry).

High level comment: It looks like you've had to write the majority of the parser from scratch (i.e. have your own representations of Module / Class / etc.) The pdoc.import_module behavior seems to just be a fancy AST parser?

Yeah, I think I'm probably going to drop pdoc and just use inspect.

Also, @dhermes for the ci bot, should I just squash those commits? I didn't want to leave out the log of their contributions.

I don't particularly mind about the CI, though those initial commits aren't necessarily needed for your work.

Nice -- this is awesome. Thanks for cranking on it.

CLAs look good, thanks!

We found a Contributor License Agreement for you (the sender of this pull request) and all commit authors, but as best as we can tell these commits were authored by someone else. If that's the case, please add them to this pull request and have them confirm that they're okay with these commits being contributed to Google. If we're mistaken and you did author these commits, just reply here to confirm.

The "continuation line" lint failures are obnoxious: finding the style which passes both pep8 and pylint is always a bit of trial and error for me. The others are fairly straightforward fixes.

Ok thanks! Should we trim the TOC back a bit?

@dhermes, I'll fix that formatting issue and I also just noticed that I broke the sub nav links with my last push.

Ok! Have another look through and LMKWYT

http://daspecster.github.io/gcloud-python/

Sorry, forgot to tag @tseaver and @dhermes on that last comment.

The docstring examples on the datastore landing page still look borked:

http://daspecster.github.io/gcloud-python/#/docs/master/gcloud/datastore

The snippet captions appear to have regressed:

http://daspecster.github.io/gcloud-python/#/docs/master/gcloud/bigquery

http://daspecster.github.io/gcloud-python/#/docs/master/gcloud/pubsub

Testing deploy with 1:1 class:json. Add overview.html Deploy script working now in testing. Fix code blocks. Add href support. Add snippets to base service module. closes googleapis#1951 Fix types generation. Fix linking. Fix external github links. Update gcloud-commit site files. Swtich to live site generation. clean up manifest for starters. Remove old example detection code. Update TOC.

I think I somehow squashed the snippet title code out. I'll try and redo it real quick.

@tseaver fixed! http://daspecster.github.io/gcloud-python/#/docs/master/gcloud/pubsub
I fixed the .. doctest:: thing with #2006.

Not sure how that got lost, I think I may have lost the commit in rebasing or something.

@daspecster The http://daspecster.github.io/gcloud-python/stable/bigquery-usage.html page is still showing the pre-snippet examples: maybe that will get fixed when you merge?

@tseaver Oh sorry, I pushed gh-pages when I was doing #2009 and #2008.
It's back now though. http://daspecster.github.io/gcloud-python/#/docs/master/gcloud/pubsub

Let me know if there is anything that's a show stopper here. I would like to try and get this merged today if possible.

LGTM

I'm going to merge this at 2pm EST unless I hear otherwise. Just to give everyone a chance to chime in.

Travis master build that I'm watching for this https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/146717128

…_wait API (#1775) * docs: update query destination table legacy file to use query_and_wait API * docs: update the query_destination_table_legacy.py to use the query_and_wait API --------- Co-authored-by: Salem Boyland <salemb@google.com> Co-authored-by: Kira <kirnendra@google.com> Co-authored-by: Chalmer Lowe <chalmerlowe@google.com>

…_many snippet (#1775) This PR adds argparse support to the download_many snippet for CLI testing, and updates the description containing traversal safety.

daspecster assigned dhermes May 5, 2016

daspecster added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label May 5, 2016

googlebot added the cla: no This human has *not* signed the Contributor License Agreement. label May 5, 2016

dhermes reviewed May 5, 2016
View reviewed changes

jgeewax reviewed May 5, 2016
View reviewed changes

Comment thread scripts/generate_json_docs.py Outdated

def generate_doc_types_json(modules, types_file_path):

doc_types_list = [{

"id": "gcloud",

This comment was marked as spam.

Sign in to view

jgeewax added the docs label May 5, 2016

daspecster force-pushed the json-docs branch from ad8fbe2 to 4db8723 Compare May 5, 2016 22:07

googlebot added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels May 5, 2016

tseaver mentioned this pull request May 6, 2016

Avoid get / set of IAM policy if current account doesn't have permission #1782

Merged

googlebot added cla: no This human has *not* signed the Contributor License Agreement. and removed cla: yes This human has signed the Contributor License Agreement. labels Jun 6, 2016

daspecster force-pushed the json-docs branch 3 times, most recently from 7610b13 to 820f2ac Compare July 20, 2016 18:46

daspecster added 3 commits July 20, 2016 15:23

First pass at Python doc generator...(originally JJ's commit)

45f5189

Update site and fix typos.

803266b

daspecster force-pushed the json-docs branch from 820f2ac to 803266b Compare July 20, 2016 19:23

daspecster mentioned this pull request Jul 20, 2016

Remove unused doctest directive. #2006

Merged

Add RST snippet captions.

42b1eb1

Remove old gh-pages doc build script.

3ef7819

daspecster merged commit ba6b13c into googleapis:master Jul 22, 2016

This was referenced Aug 3, 2016

Prepare 0.18.0 release #2044

Closed

Prepare 0.18.0 release. #2048

Merged

daspecster deleted the json-docs branch January 24, 2017 15:18

theacodes unassigned dhermes Sep 28, 2018

+              def generate_doc_types_json(modules, types_file_path):
+                  doc_types_list = [{
+                      "id": "gcloud",

		@@ -0,0 +1,160 @@
		{

WIP: Json documentation generation by daspecster · Pull Request #1775 · googleapis/google-cloud-python · GitHub

Uh oh!

WIP: Json documentation generation - #1775

WIP: Json documentation generation#1775daspecster merged 5 commits intogoogleapis:mastergoogleapis/google-cloud-python:masterfrom daspecster:json-docsdaspecster/google-cloud-python:json-docsCopy head branch name to clipboard

Conversation

daspecster commented May 5, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

googlebot commented May 5, 2016

Uh oh!

dhermes commented May 5, 2016

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

dhermes commented May 5, 2016

Uh oh!

daspecster commented May 5, 2016

Uh oh!

daspecster commented May 5, 2016

Uh oh!

dhermes commented May 5, 2016

Uh oh!

This comment was marked as spam.

Uh oh!

jgeewax commented May 5, 2016

Uh oh!

googlebot commented May 5, 2016

Uh oh!

googlebot commented Jun 6, 2016

Uh oh!

tseaver commented Jun 6, 2016

Uh oh!

daspecster commented Jul 16, 2016

Uh oh!

daspecster commented Jul 18, 2016

Uh oh!

daspecster commented Jul 20, 2016

Uh oh!

daspecster commented Jul 20, 2016

Uh oh!

tseaver commented Jul 20, 2016

Uh oh!

daspecster commented Jul 20, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daspecster commented Jul 21, 2016

Uh oh!

tseaver commented Jul 21, 2016

Uh oh!

daspecster commented Jul 21, 2016

Uh oh!

daspecster commented Jul 22, 2016

Uh oh!

tseaver commented Jul 22, 2016

Uh oh!

daspecster commented Jul 22, 2016

Uh oh!

daspecster commented Jul 22, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

WIP: Json documentation generation#1775
daspecster merged 5 commits into
googleapis:masterfrom
daspecster:json-docs

daspecster commented May 5, 2016 •
edited
Loading

daspecster commented Jul 20, 2016 •
edited
Loading