Repository navigation
document where nightly wheels are published #9994
Description
It would be great to have a nightly pip package as provided by tensorflow such that users can easily experiment with the most recent code without having to build from source, e.g. publish a package matplotlib-nightly to pypi as part of builds on master.
I came across this document from 2015 discussing a similar idea, but couldn't find anything on a nightly package in the installation instructions.
Activity
matthew-brett commented on Dec 13, 2017
Actually, we're already doing this:
Any interest in putting a PR to the install instructions?
tillahoffmann commented on Dec 13, 2017
Oh, great news! Is that URL stable? Happy to send a PR.
matthew-brett commented on Dec 13, 2017
Yes, the URL is stable (I know it doesn't look like it)...
tacaswell commented on Dec 13, 2017
Is there any plan to cull those over time?
matthew-brett commented on Dec 13, 2017
No plan - but we could do that ...
jklymak commented on Dec 13, 2017
Is it possible to have a more legible/canonical redirect somewhere?
matthew-brett commented on Dec 13, 2017
It's canonical, but not legible I agree. I don't personally know of any free services that provide https redirects - anyone have any good ideas? It has to be https in order for pip to use the URL without cumbersome extra flags.
tacaswell commented on Dec 13, 2017
We use cloudflare for https on matplotlib.org ( @jkseppan took care of setting that up ). I wonder if we can use that? https://nightly.matplotlib.org ? I am not sure if that is possible given the way that we are using a custom domain with github pages...
matthew-brett commented on Dec 13, 2017
That would be very good - if someone could configure that.
jkseppan commented on Dec 14, 2017
I set up a CNAME record at nightly.matplotlib.org pointing to 7933911d6844c6c53a7d-47bd50c35cd79bd838daf386af554a83.ssl.cf2.rackcdn.com (@tcaswell may have received a security warning about that) but that doesn't seem to be enough. I couldn't enable Cloudflare's proxying for that address (I suspect they blacklist other CDNs) but the CNAME should work anyway. Perhaps there is some additional step to take on the Rackspace side by whoever has the credentials to do that?
Rackspace instructions indicate that SSL is not going to work with CNAME records. It seems to me that the requirements of a custom domain and SSL support are not compatible with the free-of-charge solution of just adding DNS records at Cloudflare.
This would probably need a very small Heroku app (or similar) to return "302 Redirect" responses at a custom domain, pointing to the rackcdn address. But at least on Heroku the SSL support is only available on paid plans.
jkseppan commented on Dec 14, 2017
The error message at http://nightly.matplotlib.org seems to be an Akamai message, so if anyone has access to the Akamai (or possibly Rackspace) account used to configure the rackcdn.com subdomain, it probably needs to be set up to allow the CNAME redirection. But, again, that will not work with SSL.
tillahoffmann commented on Dec 14, 2017
Would it be possible to publish the wheel to pypi under a different name such as matplotlib-nightly? That would avoid redirect difficulties and allow users to install the nightly build without having to specify a specific URL.
tacaswell commented on Dec 14, 2017
I do not think that pip can express 'conflicts with' in the requirements which means you could install both matplotlib and matplotlib-nightly which would overlap and would lead to interesting bug reports!
I would much rather work with in the system (here using non-canonical repositories and the '--pre' flag) rather than work around the system (by creating a package with a slightly different name).
1 remaining item
jkseppan commented on Dec 14, 2017
There could be some danger in using Cloudflare's services for something that could get used in automated scripts. I think Cloudflare is mainly intended for serving web pages to humans efficiently, and if they detect what looks like a distributed attack they could start serving CAPTCHA challenges or otherwise limiting the usage.
tacaswell commented on Dec 14, 2017
Sweet!
Somewhat fortuitously, I will be having a chat with someone from cloudflare in the near future anyway (we push enough data every month they noticed us). I'll be sure to bring this up.
matthew-brett commented on Mar 28, 2021
Where are we on nightly wheels? Anything I can do to help?
tacaswell commented on Mar 28, 2021
We have moved to building the wheels via github actions (see https://github.com/matplotlib/matplotlib/actions/workflows/cibuildwheel.yml). There is a download link, but I am not sure if that is something you can easily point pip at.
matthew-brett commented on Mar 28, 2021
Sorry - I am sure I'm being dumb but I couldn't see how to get the wheels from that interface.
But - if we can't point pip at the results - is there a way to upload them them somewhere pip can see? I think the desire is to have something we can test our individual CIs against - certainly I would like to do that.
tacaswell commented on Mar 29, 2021
If you click on the PR tittle you get to https://github.com/matplotlib/matplotlib/actions/runs/695524406 which has a download link at the bottom (sorry, typed this 5 hours ago and failed to hit 'comment' 🤦 ).
matthew-brett commented on Mar 29, 2021
Aha - thanks - Firefox deceived me and I couldn't see or easily get to the bottom of the page.
So, the individual jobs generate a .zip file containing the wheels. Could we make a cron job or similar to download the zip file, unpack and upload somewhere like Anaconda.org?
QuLogic commented on Mar 29, 2021
See MacPython/matplotlib-wheels#15
dstansby commented on May 4, 2021
Just a quick note that we would be very interested in nightly wheels of Matplotlib over at sunpy too - currently we test against the Matplotlib master branch.