A blog by Devendra Tewari
We have this really big list of projects in a GitLab group/namespace hosted at—let’s say—https://example.com/groups/bigone. I wanted to clone all the projects there, and do some additional housekeeping in the cloned repository such as configure the email address, and add another remote location.
Now, there are easy ways to do this using the GitLab API and some coding, but we’re trying to do this the hard way, remember?
Version 4 of the GitLab API can be accessed at https://example.com/api/v4.
The first step is to find the id of the group we’re interested in
https://example.com/api/v4/groups?search=bigone
Next, we list all the projects in the group using its id, 84 in this case
https://example.com/api/v4/groups/84/projects?simple=true&order_by=name&sort=asc
I copied the JSON returned by the above query to Code, and reformatted it so it looks like
[
{
"id": 846,
"description": "Project 1",
"name": "project1",
"name_with_namespace": "bigone / project1",
"path": "project1",
"path_with_namespace": "bigone/project1",
"created_at": "2018-02-15T15:32:17.966Z",
"default_branch": "master",
"tag_list": [],
"ssh_url_to_repo": "git@example.com:bigone/project1.git",
"http_url_to_repo": "https://example.com/bigone/project1.git",
"web_url": "https://example.com/bigone/project1",
"avatar_url": null,
"star_count": 0,
"forks_count": 0,
"last_activity_at": "2018-02-16T20:48:08.377Z"
},
{
"id": 680,
"description": "Project 2",
"name": "project2",
"name_with_namespace": "bigone / project2",
"path": "project2",
"path_with_namespace": "bigone/project2",
"created_at": "2017-10-25T14:29:55.195Z",
"default_branch": null,
"tag_list": [],
"ssh_url_to_repo": "git@example.com:bigone/project2.git",
"http_url_to_repo": "https://example.com/bigone/project2.git",
"web_url": "https://example.com/bigone/project2",
"avatar_url": null,
"star_count": 0,
"forks_count": 0,
"last_activity_at": "2017-10-25T14:29:55.195Z"
},
...
The above query only returns the first page, so I repeated the above query a few times—incrementing the page parameter—until I had a complete list of projects
https://example.com/api/v4/groups/84/projects?simple=true&order_by=name&sort=asc&page=2
Next, I used the JSON Transform plugin of Code to transform—with [].name—the above JSON to
[
"project1",
"project2",
...
]
Next, I used regular expression search—^ "(.*)",—and replace—git clone git@example.com:bigone/$1.git; cd $1; git remote add other git@other.com:bigone/$1.git; git config user.email me@example.com; cd ..—and some editing, to produce this
#!/bin/bash
git clone git@example.com:bigone/project1.git; cd project1; git remote add other git@other.com:bigone/project1.git; git config user.email me@example.com; cd ..
git clone git@example.com:bigone/project2.git; cd project2; git remote add other git@other.com:bigone/project2.git; git config user.email me@example.com; cd ..
...
That was all I needed to clone all the projects in the big GitLab group, and set them up properly.
To wrap it up, here’s the command I run to sync all the remotes
find . -maxdepth 1 -not -path . -not -path '*/\.*' -exec sh -c 'echo {}; cd {}; git pull; git push other master' \;