@tseaver PTAL

@tseaver PTAL

@tseaver Only comments on the doctest itself / associated repr() changes.

Any comments on the approach of making a fake docs directory and then running doctest there?

Any comments on the approach of making a fake docs directory and then running doctest there?

We're going to need a way to test all the examples, not just those embedded in docstrings: do you have a plan for that?

@dhermes Memory eludes me: what stops us from just running sphinx-build -b doctest in place in the normal docs directory, rather than synthesizing one?

We're going to need a way to test all the examples, not just those embedded in docstrings: do you have a plan for that?

Yes, though that is "beyond" the scope of this PR. There are two possible approaches that I can see. The first is to "guess" which hand-written RST files apply to a given package (I am not "pro" on this approach). The second is as follows:

• Discontinue handwritten RST files and use sphinx-apidoc to generate our docs (I wrote the docs gen for oauth2client using this and @jonparrott "copied" that approach in https://github.com/GoogleCloudPlatform/google-auth-library-python/)
  • In this world, our "package usage" docs will go in google/cloud/foo/__init__.py
• Add checks that literal blocks are as we expect, e.g. no python literal blocks that aren't tested

Memory eludes me: what stops us from just running sphinx-build -b doctest in place in the normal docs directory, rather than synthesizing one?

The synthesized one is an attempt to isolate on a per-package basis (just as is done with the system tests). It's not strictly necessary.

The "real" reason I did it was because our docs build takes quite a long time (since the project is so large), and I wanted to have a snappier doctest build.

@tseaver Can we keep moving on this?

@dhermes I'd still say that running sphinx-build -b doctest (maybe in a separate [testenv:doctest] environment w/ appropriate system-tests deps) would be the right way to move forward: it will pick up any snippets in non-autodoc files, for instance.

I'd still say that running sphinx-build -b doctest (maybe in a separate [testenv:doctest] environment w/ appropriate system-tests deps) would be the right way to move forward: it will pick up any snippets in non-autodoc files, for instance.

This is fine, though I see it having two issues:

• We can't cherry pick which packages we run it for (this is actually really nasty because the docs build is very slow)
• We have to set up and teardown system test-like behavior yet again in CI (as opposed to the scenario where doctests run with the system tests)

Plan moving forward:

• use sphinx-apidoc for most RST files
• leave usage docs in handwritten RST files
• make a (potentially "bit-rot"-able) script to pick up usage docs along with other package files when running isolated doctest
• write TOC by hand

LGTM.