About the batch processing script of the warehouse, the efficiency is improved by 500%

Leave a Comment / / Technical house | 技术小黑屋

Many times, we will encounter such a scene

  • Changed a new computer, need to clone gitlab repos one by one?
  • Not sure which repo contains the maven.aliyun.com setting?
  • Can I update local repos in batches?

If you have the above questions or situations, you can try some methods of batch processing repos in this article

batch clone

 1 
ruby cloneRepos.rb code-git-xxxxxx ../projects/

The content of cloneRepos.rb script is as follows

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 twenty one twenty two twenty three twenty four 25 
#!/usr/bin/env ruby # encoding: utf-8 require 'httparty' require 'json'  def cloneRepos ( repoUrlApi ) headers = { 'PRIVATE-TOKEN' : ARGV [ 0 ] , } response = HTTParty . get ( repoUrlApi , headers : headers ) data = JSON . parse ( response . body ); puts data . length () data . each { | e | name = e [ 'name' ] gitUrl = e [ 'ssh_url_to_repo' ] system "cd #{ ARGV [ 1 ] } && git clone #{ gitUrl } " puts name } end  cloneRepos ( 'https://code.hahaha.io/api/v4/projects?per_page=100' ) cloneRepos ( 'https://code.hahaha.io/api/v4/projects?per_page=100&page=2' )

Parameter explanation

  • code-git-xxxxxx The token obtained from gitlab is obtained according to the instructions in the following figure

https://asset.droidyue.com/image/2022/h2/QQ20220904-220957%402x.png

  • The directory where ../projects is stored

Notice

This script can only process the first 200 repos at present. If necessary, you can modify the code to process it yourself.
Batch project search

quick search

For example, we want to search maven.aliyun.com

 1 2 3 4 5 6 7 8 9 
projects gradleSearch.sh maven.aliyun.com ./xxxx/example/android/build.gradle:7: url 'http://maven.aliyun.com/nexus/content/repositories/releases/' ./xxxxx/example/android/build.gradle:23: url 'http://maven.aliyun.com/nexus/content/repositories/releases/' ./xxxx/android/build.gradle:22: url 'https://maven.aliyun.com/repository/public/' ./xxxx/example/android/build.gradle:6: maven { url 'https://maven.aliyun.com/repository/google'

The content of gradleSearch.sh is as follows

 1 2 
#!/bin/bash find . -name "*.gradle" | xargs grep -E -n --color = always -r "$1"

Bulk update

 1 
updateRepo.sh

Its content is like this

 1 2 3 4 5 6 7 8 9 10 
#!/bin/bash for dir in */; do    echo "$dir" realpath = ` realpath $dir ` echo $realpath cd $realpath git checkout master git pull origin master cd - done

Notice

  • If the current repo has uncommitted changes, it cannot be updated.

Through the above few scripts, we can easily achieve efficiency improvement.

droidyue_gzh_green_png.png

This article is reprinted from https://droidyue.com/blog/2022/09/04/speed-up-about-cloning-git-repos-and-updating-them/
This site is for inclusion only, and the copyright belongs to the original author.

Leave a Comment