Repository navigation
gl.projects.get() fails when provided "namespace/project_name" #2130
Description
Activity
- changed the title [-]Can't gl.projects.get with project_with_namespace[/-] [+]Can't gl.projects.get with namespace/project_name[/+] on Jul 6, 2022
- changed the title [-]Can't gl.projects.get with namespace/project_name[/-] [+]gl.projects.get() fails when provided "namespace/project_name"[/+] on Jul 6, 2022
Don't know if this is also helpful.
% gitlab -c ~/.python-gitlab.cfg -g gitlab-https project-notification-settings get --project-id https-phase2/ha-proxy-migration Impossible to get object (gitlab.v4.objects.notification_settings.NotificationSettingsManager.get() got multiple values for keyword argument 'id')
Hi @walterrowe
I know that it does work to get by the path as I use that often. So this is odd.
Can you add a print statement so we see the name of the project?
for this_project in options.project: print(f"Project: {this_project!r}") this_project = my_gitlab.projects.get(this_project) print("Adding project:", this_project.path_with_namespace) projects = projects + [this_project]See the first line of output for the failure. The last command line arg is the project with name space.
See the first line of output for the failure. The last command line arg is the project with name space.
But can you print the value that you are sending to the get() method please. I would like to see. One thought I had is that could possibly be iterating over the string and thus the project trying to be retrieved could be the letter h. Printing the value will help figure that out.
Output showing value and type ...
% ./project-status.py --config ~/.python-gitlab.cfg --section gitlab-https --project https-phase2/ha-proxy-migration Looking for project: 'https-phase2/ha-proxy-migration' (type: <class 'str'>) Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/gitlab/exceptions.py", line 330, in wrapped_f return f(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/gitlab/mixins.py", line 139, in get server_data = self.gitlab.http_get(path, **kwargs) File "/usr/local/lib/python3.10/site-packages/gitlab/client.py", line 790, in http_get result = self.http_request( File "/usr/local/lib/python3.10/site-packages/gitlab/client.py", line 756, in http_request raise gitlab.exceptions.GitlabHttpError( gitlab.exceptions.GitlabHttpError: 404: 404 Not Found The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/wrowe/git-repos/python-gitlab/./project-status.py", line 140, in <module> this_project = my_gitlab.projects.get(this_project) File "/usr/local/lib/python3.10/site-packages/gitlab/v4/objects/projects.py", line 775, in get return cast(Project, super().get(id=id, lazy=lazy, **kwargs)) File "/usr/local/lib/python3.10/site-packages/gitlab/exceptions.py", line 332, in wrapped_f raise error(e.error_message, e.response_code, e.response_body) from e gitlab.exceptions.GitlabGetError: 404: 404 Not Found
argparser code that processes the command line option for project.
group1.add_argument( "--project", dest="project", type=str, help="list of one or more project paths or ids", nargs="+", action="extend", )
Looking in mixins.py I see that GetMixin calls utils.EncodedID.
python-gitlab/gitlab/mixins.py
Lines 130 to 131 in d5de4b1
utils.py EncodedID() has these lines:
Lines 127 to 128 in d5de4b1
in a terminal window I can type this and it works.
>>> import urllib.parse >>> urllib.parse.quote("https-phase2/ha-proxy-migration",safe="") 'https-phase2%2Fha-proxy-migration'
>>> this_project="https-phase2/ha-proxy-migration" >>> urllib.parse.quote(this_project,safe="") 'https-phase2%2Fha-proxy-migration'
Very odd 😕
You can add this to your program after getting the gl=gitlab.Gitlab() value.
gl.enable_debug()
This will output the data that is being sent to the server.
Looking in mixins.py I see that GetMixin calls utils.EncodedID.
python-gitlab/gitlab/mixins.py
Lines 130 to 131 in d5de4b1
The quoting is working as I expected:
>>> import gitlab.utils as utils >>> a = "https-phase2/ha-proxy-migration" >>> utils.EncodedId(a) 'https-phase2%2Fha-proxy-migration'
with debug enabled ...
% ./project-status.py --config ~/.python-gitlab.cfg --section gitlab-https --project https-phase2/ha-proxy-migration Looking for project: 'https-phase2/ha-proxy-migration' (type: <class 'str'>) DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): my.gitlab.server:443 send: b'GET /api/v4/projects/https-phase2%2Fha-proxy-migration HTTP/1.1\r\nHost: my.gitlab.server\r\nUser-Agent: python-gitlab/3.6.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nPRIVATE-TOKEN: my_private_token\r\nContent-type: application/json\r\n\r\n' reply: 'HTTP/1.1 404 Not Found\r\n' header: Server: nginx header: Date: Wed, 06 Jul 2022 17:07:29 GMT header: Content-Type: application/json header: Content-Length: 25 header: Connection: keep-alive header: Cache-Control: no-cache header: Vary: Origin header: X-Content-Type-Options: nosniff header: X-Frame-Options: SAMEORIGIN header: X-Request-Id: 01G7A7P9ERX4PV59KBYQF1SZ17 header: X-Runtime: 0.008983 DEBUG:urllib3.connectionpool:https://my.gitlab.server:443 "GET /api/v4/projects/https-phase2%2Fha-proxy-migration HTTP/1.1" 404 25 [ the rest of the traceback is the same as above ]
Also if you want to hop on the Gitter channel I could try to help in a more interactive way. https://gitter.im/python-gitlab/Lobby
Would love the interactive help but I can't connect to the gitter.im channel from this restricted network.
One thing I was reading was this:
https://docs.gitlab.com/ee/api/#namespaced-path-encodingA project’s path isn’t necessarily the same as its name. A project’s path is found in the project’s URL or in the project’s settings, under General > Advanced > Change path.
But it looks like it is correct 😕
Can you try this?
this_project = my_gitlab.projects.get(3894 ) print(f"{this_project.path_with_namespace!r}")Want to make sure there isn't some extra unseen characters in the path.
Another possibility is that Gitlab 14.0 has a bug, but I wouldn't think so.
with debug enabled ...
% ./project-status.py --config ~/.python-gitlab.cfg --section gitlab-https --project https-phase2/ha-proxy-migration Looking for project: 'https-phase2/ha-proxy-migration' (type: <class 'str'>) DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): my.gitlab.server:443 send: b'GET /api/v4/projects/https-phase2%2Fha-proxy-migration HTTP/1.1\r\nHost: my.gitlab.server\r\nUser-Agent: python-gitlab/3.6.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nPRIVATE-TOKEN: my_private_token\r> ```
The path being sent to Gitlab looks correct to me.
send: b'GET /api/v4/projects/https-phase2%2Fha-proxy-migration
The program does that.
% ./project-status.py --config ~/.python-gitlab.cfg --section gitlab-https --project 3894 Looking for project: '3894' (type: <class 'str'>) Adding project: https-phase2/ha-proxy-migration Scanning issues for ha-proxy-migration ... Building report for ha-proxy-migration ... Writing report file ...
The program does that.
% ./project-status.py --config ~/.python-gitlab.cfg --section gitlab-https --project 3894 Looking for project: '3894' (type: <class 'str'>) Adding project: https-phase2/ha-proxy-migration Scanning issues for ha-proxy-migration ... Building report for ha-proxy-migration ... Writing report file ...
I don't see quotes around the project name in "Adding project" can you do it with
print(f"{this_project.path_with_namespace!r}")Yes.
% ./project-status.py --config ~/.python-gitlab.cfg --section gitlab-https --project 3894 Looking for project: '3894' (type: <class 'str'>) Adding project: 'https-phase2/ha-proxy-migration' Scanning issues for ha-proxy-migration ... Building report for ha-proxy-migration ... Writing report file ...
the python script snippet:
for this_project in options.project: print(f"\nLooking for project: {this_project!r} (type: {type(this_project)})\n") this_project = my_gitlab.projects.get(this_project) print(f"Adding project: {this_project.path_with_namespace!r}") projects = projects + [this_project]
is it possible this is a bug in the 14.0.12 CE API? we are upgrading to 15.x over time. gitlab upgrades require iterating over each version unfortunately.
Sadly I don't know. It seems like python-gitlab is sending the correct path to the Gitlab server based on the debug output.
The only other shot in the dark I have is to also print the url-encoded ID form.
encoded_id = gitlab.utils.EncodedId(this_project.path_with_namespace) print(f"{encoded_id!r}")And compare to what trying to send.
is it possible this is a bug in the 14.0.12 CE API? we are upgrading to 15.x over time. gitlab upgrades require iterating over each version unfortunately.
Possible yes. I don't know though. I would think it isn't as it seems like a big bug. But not sure.
Also. Is this failing for all/other projects when doing a get using a path?
RESOLVED:
On this specific gitlab instance, I had to use https://my.gitlab.server/toplevel_entrypath for url in ~/.python-gitlab.cfg
This fails:
[gitlab-https] url = https://my.gitlab.server/ private_token = my_private_token api_version = 4
This works:
[gitlab-https] url = https://my.gitlab.server/gitlab private_token = my_private_token api_version = 4
We don't have this "toplevel_entrypath" layer in our paths on our other gitlab instances. I need to talk to her gitlab admin about that.
Reacted by John VillalovosWoo hoo!! Glad you figured it out. I was perplexed for sure.
Just curious. Does the web_url attribute show the correct path? Maybe next time I will ask to see that value too 🤓
Also confusing to me why the integer ID worked. Weird.
The integer ID probably worked because gitlab searches a different field in the database for an ID versus for a namespace/project_name.
and since you asked ...
Adding project: 'https-phase2/ha-proxy-migration' ('https://my.gitlab.server/gitlab/https-phase2/ha-proxy-migration')
definitely worth also asking for the web_url in the future.
Reacted by John VillalovosThe integer ID probably worked because gitlab searches a different field in the database for an ID versus for a namespace/project_name.
and since you asked ...
Adding project: 'https-phase2/ha-proxy-migration' ('https://my.gitlab.server/gitlab/https-phase2/ha-proxy-migration')
definitely worth also asking for the web_url in the future.
Thanks. I'll try to remember that!
- locked as resolved and limited conversation to collaborators on Jul 10, 2023
Description of the problem, including code/CLI snippet
In project-status.py ...
Expected Behavior
this_project = my_gitlab.projects.get("namespace/project_name") succeeds
Actual Behavior
this_project = my_gitlab.projects.get("namespace/project_name") fails with 404 error
Specifications