Hello. The bot has been working great until recently. I noticed at astropy/astropy#13108 that it is not triggered by the backport-v5.0.x label anymore, which used to work. Can you please check your bot logs and let me know what is going on? Has the procedure changed?
2022-04-19T13:41:38.240816+00:00 app[web.1]: Making a GET call to https://api.github.com/repos/astropy/astropy/pulls/13113
2022-04-19T13:41:38.567610+00:00 app[web.1]: Making a GET call to https://api.github.com/repos/astropy/astropy/collaborators/pllim/permission
2022-04-19T13:41:38.780492+00:00 app[web.1]: found permission admin for user pllim on astropy astropy
2022-04-19T13:41:38.780593+00:00 app[web.1]: :: treating backport to v5.0.x
2022-04-19T13:41:38.862691+00:00 app[web.1]: :: testing who can use <function safe_backport at 0x7fa5f66e8160>
2022-04-19T13:41:38.862787+00:00 app[web.1]: Making a GET call to https://api.github.com/repos/astropy/astropy/contents/.meeseeksdev.yml
2022-04-19T13:41:39.084845+00:00 app[web.1]: config file not found
2022-04-19T13:41:39.084893+00:00 app[web.1]: Custom allowed command for pllim are []
2022-04-19T13:41:39.084935+00:00 app[web.1]: with everyone taken into account []
2022-04-19T13:41:39.084960+00:00 app[web.1]: would not allow pllim to backport
2022-04-19T13:41:39.085121+00:00 app[web.1]: :: authorisation granted Permission.write custom_rule: False {}
2022-04-19T13:41:39.085332+00:00 app[web.1]: Making a GET call to https://api.github.com/repos/astropy/astropy
2022-04-19T13:41:39.308517+00:00 app[web.1]: Making a GET call to https://api.github.com/repos/astropy/astropy/branches
2022-04-19T13:41:39.535801+00:00 app[web.1]: [backport] found branch v5.0.x
2022-04-19T13:41:39.535831+00:00 app[web.1]: [backport] == Collecting data on Pull-request ...
2022-04-19T13:41:39.535996+00:00 app[web.1]: Making a GET call to https://api.github.com/repos/astropy/astropy/pulls/13113
2022-04-19T13:41:39.892752+00:00 app[web.1]: [backport] ----------------------------------------
2022-04-19T13:41:39.892764+00:00 app[web.1]: [backport] ----------------------------------------
2022-04-19T13:41:40.097287+00:00 app[web.1]: Making a POST call to https://api.github.com/repos/astropy/astropy/issues/13113/comments
2022-04-19T13:41:40.960599+00:00 app[web.1]: [backport] Something went wrong
2022-04-19T13:41:40.960683+00:00 app[web.1]: [backport] 401 Client Error: Unauthorized for url: https://api.github.com/repos/astropy/astropy/forks
2022-04-19T13:41:41.054205+00:00 app[web.1]: ERROR:tornado.application:Uncaught exception POST /webhook (10.1.31.217)
2022-04-19T13:41:41.054208+00:00 app[web.1]: HTTPServerRequest(protocol='http', host='meeseeksbot.herokuapp.com', method='POST', uri='/webhook', version='HTTP/1.1', remote_ip='10.1.31.217')
2022-04-19T13:41:41.054209+00:00 app[web.1]: Traceback (most recent call last):
2022-04-19T13:41:41.054209+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.10/site-packages/tornado/web.py", line 1702, in _execute
2022-04-19T13:41:41.054210+00:00 app[web.1]: result = method(*self.path_args, **self.path_kwargs)
2022-04-19T13:41:41.054211+00:00 app[web.1]: File "/app/meeseeksdev/meeseeksbox/core.py", line 212, in post
2022-04-19T13:41:41.054211+00:00 app[web.1]: return self.dispatch_action(action, payload)
2022-04-19T13:41:41.054211+00:00 app[web.1]: File "/app/meeseeksdev/meeseeksbox/core.py", line 298, in dispatch_action
2022-04-19T13:41:41.054211+00:00 app[web.1]: self.dispatch_on_mention(body, payload, user)
2022-04-19T13:41:41.054212+00:00 app[web.1]: File "/app/meeseeksdev/meeseeksbox/core.py", line 568, in dispatch_on_mention
2022-04-19T13:41:41.054212+00:00 app[web.1]: maybe_gen = handler(
2022-04-19T13:41:41.054213+00:00 app[web.1]: File "/app/meeseeksdev/meeseeksbox/commands.py", line 730, in safe_backport
2022-04-19T13:41:41.054213+00:00 app[web.1]: frk = session.personal_request(
2022-04-19T13:41:41.054213+00:00 app[web.1]: File "/app/meeseeksdev/meeseeksbox/utils.py", line 277, in personal_request
2022-04-19T13:41:41.054214+00:00 app[web.1]: response.raise_for_status()
2022-04-19T13:41:41.054214+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.10/site-packages/requests/models.py", line 960, in raise_for_status
2022-04-19T13:41:41.054214+00:00 app[web.1]: raise HTTPError(http_error_msg, response=self)
2022-04-19T13:41:41.054227+00:00 app[web.1]: requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://api.github.com/repos/astropy/astropy/forks
2022-04-19T13:41:41.054634+00:00 heroku[router]: at=info method=POST path="/webhook" host=meeseeksbot.herokuapp.com request_id=1bfad0dc-bc73-4d63-8fc1-d96d8b4e74ec fwd="140.82.115.91" dyno=web.1 connect=0ms service=3099ms status=500 bytes=274 protocol=https
2022-04-19T13:41:41.054726+00:00 app[web.1]: ERROR:tornado.access:500 POST /webhook (10.1.31.217) 3098.48ms
2022-04-19T13:41:41.262391+00:00 app[web.1]: (https://github.com/astropy/astropy/issues/13113) Not responding to self
It might be related to the recent heroku/salesforce outage. I haven't had a chance to check, but there was indeed issue with autobackport from milestone/label that recently affected IPython as well.
Activity
pllim commented on Apr 19, 2022
Hmm it is even more cryptic at astropy/astropy#13113
pllim commented on Apr 19, 2022
Thanks for the quick response. I don't grok the first log, nothing jumps out to me, does it look normal to you?
The second one has 401 Client Error: Unauthorized for url: https://api.github.com/repos/astropy/astropy/forks but I don't know what it means.
blink1073 commented on Apr 19, 2022
2022-04-19T13:41:38.240816+00:00 app[web.1]: Making a GET call to https://api.github.com/repos/astropy/astropy/pulls/13113 2022-04-19T13:41:38.567610+00:00 app[web.1]: Making a GET call to https://api.github.com/repos/astropy/astropy/collaborators/pllim/permission 2022-04-19T13:41:38.780492+00:00 app[web.1]: found permission admin for user pllim on astropy astropy 2022-04-19T13:41:38.780593+00:00 app[web.1]: :: treating backport to v5.0.x 2022-04-19T13:41:38.862691+00:00 app[web.1]: :: testing who can use <function safe_backport at 0x7fa5f66e8160> 2022-04-19T13:41:38.862787+00:00 app[web.1]: Making a GET call to https://api.github.com/repos/astropy/astropy/contents/.meeseeksdev.yml 2022-04-19T13:41:39.084845+00:00 app[web.1]: config file not found 2022-04-19T13:41:39.084893+00:00 app[web.1]: Custom allowed command for pllim are [] 2022-04-19T13:41:39.084935+00:00 app[web.1]: with everyone taken into account [] 2022-04-19T13:41:39.084960+00:00 app[web.1]: would not allow pllim to backport 2022-04-19T13:41:39.085121+00:00 app[web.1]: :: authorisation granted Permission.write custom_rule: False {} 2022-04-19T13:41:39.085332+00:00 app[web.1]: Making a GET call to https://api.github.com/repos/astropy/astropy 2022-04-19T13:41:39.308517+00:00 app[web.1]: Making a GET call to https://api.github.com/repos/astropy/astropy/branches 2022-04-19T13:41:39.535801+00:00 app[web.1]: [backport] found branch v5.0.x 2022-04-19T13:41:39.535831+00:00 app[web.1]: [backport] == Collecting data on Pull-request ... 2022-04-19T13:41:39.535996+00:00 app[web.1]: Making a GET call to https://api.github.com/repos/astropy/astropy/pulls/13113 2022-04-19T13:41:39.892752+00:00 app[web.1]: [backport] ---------------------------------------- 2022-04-19T13:41:39.892764+00:00 app[web.1]: [backport] ---------------------------------------- 2022-04-19T13:41:40.097287+00:00 app[web.1]: Making a POST call to https://api.github.com/repos/astropy/astropy/issues/13113/comments 2022-04-19T13:41:40.960599+00:00 app[web.1]: [backport] Something went wrong 2022-04-19T13:41:40.960683+00:00 app[web.1]: [backport] 401 Client Error: Unauthorized for url: https://api.github.com/repos/astropy/astropy/forks 2022-04-19T13:41:41.054205+00:00 app[web.1]: ERROR:tornado.application:Uncaught exception POST /webhook (10.1.31.217) 2022-04-19T13:41:41.054208+00:00 app[web.1]: HTTPServerRequest(protocol='http', host='meeseeksbot.herokuapp.com', method='POST', uri='/webhook', version='HTTP/1.1', remote_ip='10.1.31.217') 2022-04-19T13:41:41.054209+00:00 app[web.1]: Traceback (most recent call last): 2022-04-19T13:41:41.054209+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.10/site-packages/tornado/web.py", line 1702, in _execute 2022-04-19T13:41:41.054210+00:00 app[web.1]: result = method(*self.path_args, **self.path_kwargs) 2022-04-19T13:41:41.054211+00:00 app[web.1]: File "/app/meeseeksdev/meeseeksbox/core.py", line 212, in post 2022-04-19T13:41:41.054211+00:00 app[web.1]: return self.dispatch_action(action, payload) 2022-04-19T13:41:41.054211+00:00 app[web.1]: File "/app/meeseeksdev/meeseeksbox/core.py", line 298, in dispatch_action 2022-04-19T13:41:41.054211+00:00 app[web.1]: self.dispatch_on_mention(body, payload, user) 2022-04-19T13:41:41.054212+00:00 app[web.1]: File "/app/meeseeksdev/meeseeksbox/core.py", line 568, in dispatch_on_mention 2022-04-19T13:41:41.054212+00:00 app[web.1]: maybe_gen = handler( 2022-04-19T13:41:41.054213+00:00 app[web.1]: File "/app/meeseeksdev/meeseeksbox/commands.py", line 730, in safe_backport 2022-04-19T13:41:41.054213+00:00 app[web.1]: frk = session.personal_request( 2022-04-19T13:41:41.054213+00:00 app[web.1]: File "/app/meeseeksdev/meeseeksbox/utils.py", line 277, in personal_request 2022-04-19T13:41:41.054214+00:00 app[web.1]: response.raise_for_status() 2022-04-19T13:41:41.054214+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.10/site-packages/requests/models.py", line 960, in raise_for_status 2022-04-19T13:41:41.054214+00:00 app[web.1]: raise HTTPError(http_error_msg, response=self) 2022-04-19T13:41:41.054227+00:00 app[web.1]: requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://api.github.com/repos/astropy/astropy/forks 2022-04-19T13:41:41.054634+00:00 heroku[router]: at=info method=POST path="/webhook" host=meeseeksbot.herokuapp.com request_id=1bfad0dc-bc73-4d63-8fc1-d96d8b4e74ec fwd="140.82.115.91" dyno=web.1 connect=0ms service=3099ms status=500 bytes=274 protocol=https 2022-04-19T13:41:41.054726+00:00 app[web.1]: ERROR:tornado.access:500 POST /webhook (10.1.31.217) 3098.48ms 2022-04-19T13:41:41.262391+00:00 app[web.1]: (https://github.com/astropy/astropy/issues/13113) Not responding to selfblink1073 commented on Apr 19, 2022
Okay, so we're not responding to labels, and for some reason the second request wasn't authorized.
blink1073 commented on Apr 19, 2022
I accidentally posted an ssh token here, so I revoked all tokens for the application
blink1073 commented on Apr 19, 2022
I'm debugging right now with jupyter-server/jupyter_server#793
blink1073 commented on Apr 19, 2022
Ah, @Carreau, it looks like the token we're using has expired:
curl \ -H "Accept: application/vnd.github.v3+json" \ -u $PERSONAL_ACCOUNT_NAME https://api.github.com/user/repos Enter host password for user '...': { "message": "Bad credentials", "documentation_url": "https://docs.github.com/rest" }pllim commented on Apr 19, 2022
Thanks for looking into this! 🙏
Carreau commented on Apr 20, 2022
I put the github account creds in the jupyter shared password. I just regenerated the token, but the previous one was not expired..
$ curl -H "Accept: application/vnd.github.v3+json" -u $PERSONAL_ACCOUNT_NAME https://api.github.com/user/repos Enter host password for user 'meeseeksmachine': [ { "id": 470708511, "node_id": "R_kgDOHA5xHw", "name": "anndata", "full_name": "meeseeksmachine/anndata", "private": false, "owner": { "login": "meeseeksmachine", "id": 39504233, "node_id": "MDQ6VXNlcjM5NTA0MjMz", ...Hopefully it should work now.
blink1073 commented on Apr 20, 2022
Okay, the bot appears to be working again for backports. Let's leave this open to track label handling.
pllim commented on Apr 20, 2022
pllim commented on Apr 20, 2022
Update: label still not working but it works via comment. See astropy/astropy#13102
blink1073 commented on Apr 20, 2022
Thanks for confirming @pllim!
QuLogic commented on Apr 21, 2022
It seems like anything with labels is broken?
and then nothing is output or a backport even attempted. The backport message is in the milestone, not the labels.
Explicitly asking for a backport on matplotlib/matplotlib#22560 worked.
QuLogic commented on Apr 22, 2022
And another:
pllim commented on May 10, 2022
Hello. Seems like this is still an issue for astropy. Is it just us?
Carreau commented on May 10, 2022
It might be related to the recent heroku/salesforce outage. I haven't had a chance to check, but there was indeed issue with autobackport from milestone/label that recently affected IPython as well.
pllim commented on May 17, 2022
Label appears to work again at astropy/astropy#13257 . Whatever you did, thanks!
Carreau commented on May 18, 2022
Thanks !