Enterprise self-built GitLab code warehouse installation and basic configuration use

0 x00 Foreword####

In the open source world, there is no end!

Description: GitLab is a very good project. This is an open source project that allows users to run a project management system similar to GitHub on their own servers.

You can use GitLab to develop private projects before public release.

Official website address: https://about.gitlab.com/

Help document: https://about.gitlab.com/install/

Component reference: https://docs.gitlab.com/ce/development/architecture.html

GitLab uses a traditional open source business model, and they have two products:

Gitlab advantages:

0 x01 environment installation####

CentOS

Gitlab basic requirements: at least 4GB of free RAM to run GitLab

Gitlab installation method:

Installation process:

Step1. Use rpm to install the community version of gitlab (replace the update source and choose by yourself):

sudo dn8 install -y curl policycoreutils openssh-server wget
# You can use wget to download and install the rpm package
wget https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el8/gitlab-ce-12.9.2-ce.0.el8.x86_64.rpm
# dnf install gitlab-ce-12.9.2-ce.0.el8.x86_64.rpm
rpm -ivh gitlab-ce-12.9.2-ce.0.el8.x86_64.rpm

Step2. After the installation is complete, the following shows that the installation is successful:

Thank you for installing GitLab!
GitLab was unable to detect a valid hostname for your instance.
Please configure a URL for your GitLab instance by setting `external_url`
configuration in/etc/gitlab/gitlab.rb file.
Then, you can start your GitLab instance by running the following command:
sudo gitlab-ctl reconfigure

Step3. Modify the default configuration of gitlab

$vim /etc/gitlab/gitlab.rb
# Modify it to your own domain name or IP, in single quotes, and don’t change the previous http (and add the domain name to hosts)
external_url 'http://gitlab.weiyigeek.top'    

# Mail configuration, no mail server can close the mail service function
# gitlab_rails['smtp_enable']=false
# gitlab_rails['smtp_address']=""
# gitlab_rails['smtp_port']=587
# gitlab_rails['smtp_user_name']=""
# gitlab_rails['smtp_password']=""
# gitlab_rails['smtp_authentication']=""
# gitlab_rails['smtp_enable_starttls_auto']=
# gitlab_rails['smtp_tls']=
# gitlab_rails['gitlab_email_from']=''

Step4. After the modification is completed, the configuration needs to be reloaded and then restarted;

$ gitlab-ctl reconfigure
# [2020- 04- 15 T10:33:44+08:00] WARN: Please install an English UTF-8 locale for Chef to use, falling back to C locale and disabling UTF-8 support. #If you feel uncomfortable, just set the system language by yourself
# [2020- 04- 15 T10:33:45+08:00] WARN: Please install an English UTF-8 locale for Chef to use, falling back to C locale and disabling UTF-8 support.
# Starting Chef Client, version 14.14.29
# resolving cookbooks for run list:["gitlab"]
# Synchronizing Cookbooks:
# - redis(0.1.0)
# - package(0.1.0)
# - postgresql(0.1.0)
# - mattermost(0.1.0)
# - consul(0.1.0)
# - letsencrypt(0.1.0)
# - gitlab(0.0.1)
# - runit(4.3.0)
# - monitoring(0.1.0)
# - praefect(0.1.0)
# - gitaly(0.1.0)
# - registry(0.1.0)
# - acme(4.1.1)
# - nginx(0.1.0)
# - crond(0.1.0)
# Running handlers:
# Running handlers complete
# Chef Client finished,541/1460 resources updated in03 minutes 14 seconds
# gitlab Reconfigured!

$ gitlab-ctl restart
# [2020- 04- 15 T10:37:51+08:00] WARN: Please install an English UTF-8 locale for Chef to use, falling back to C locale and disabling UTF-8 support.
# ok: run: alertmanager:(pid 21410) 1s
# ok: run: gitaly:(pid 21422) 0s
# ok: run: gitlab-exporter:(pid 21429) 1s
# ok: run: gitlab-workhorse:(pid 21444) 0s
# ok: run: grafana:(pid 21463) 0s
# ok: run: logrotate:(pid 21474) 0s
# ok: run: nginx:(pid 21480) 0s
# ok: run: node-exporter:(pid 21563) 1s
# ok: run: postgres-exporter:(pid 21569) 0s
# ok: run: postgresql:(pid 21580) 0s
# ok: run: prometheus:(pid 21589) 0s
# ok: run: redis:(pid 21601) 0s
# ok: run: redis-exporter:(pid 21708) 0s
# ok: run: sidekiq:(pid 21716) 0s
# ok: run: unicorn:(pid 21727) 0s

Step5. If reconfigure fails, you need to systemctl enable gitlab-runsvdir && systemctl restart gitlab-runsvdir to restart the gitlab-runsvdir service

Step6. Open the browser to initialize the account and set the password. This password is the password of the root administrator account. After setting the password, it will automatically jump to the login page.

application:http://gitlab.weiyigeek.top/
account number:root
password:WeiyiGeek #8 and above

WeiyiGeek.

Step7. The login interface is successful, is there a crazy urge to write code;

WeiyiGeek.

Step8. English may not be friendly to novices who are not good at English, we will Chineseize (Note: version higher than 12.3.5 does not require Chineseization, directly set the custom language in the user settings to select simplified Chinese);

# Stop gitlab
gitlab-ctl stop

# Get the currently installed version patch
git clone https://gitlab.com/xhang/gitlab.git
cd gitlab
# View all branch versions
git branch -a
gitlab_version=$(cat /opt/gitlab/embedded/service/gitlab-rails/VERSION)
# Compare the Chinese label with the original label, export the diff file for patch
# git diff v${gitlab_version} v${gitlab_version}-zh >../${gitlab_version}-zh.diff
git diff remotes/origin/12-3-stable remotes/origin/12-3-stable-zh >../${gitlab_version}-zh.diff

# When patching, it will prompt that some patch files do not exist. Be sure to skip these files. Otherwise, you will get an error when you reconfig.
patch -d /opt/gitlab/embedded/service/gitlab-rails -p1 <../${gitlab_version}-zh.diff

# Recompile and start
gitlab-ctl reconfigure
gitlab-ctl start
Docker

Description: Adopt a low-privileged user to manage docker and deploy gitlab service with docker;

Environmental description:

$lsb_release -a
Description:    Ubuntu 20.04.1 LTS
Release:20.04
Codename:       focal

$docker version
Client: Docker Engine - Community
 Version:19.03.13
 API version:1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 1617:02:522020
 OS/Arch:           linux/amd64
 Experimental:false

Server: Docker Engine - Community
 Engine:
 Version:19.03.13
 API version:1.40(minimum version 1.12)
 Go version:       go1.13.15
 Git commit:       4484c46d9d
 Built:            Wed Sep 1617:01:202020
 OS/Arch:          linux/amd64
 Experimental:false
 containerd:
 Version:1.3.7
 GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
 Version:1.0.0-rc10
 GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
 Version:0.18.0
 GitCommit:        fec3683

Operating procedures:

# 1. Create a new low-privileged user and add the user to the docker group
sudo useradd -m -g docker WeiyiGeek-gitlab && sudo passwd WeiyiGeek-gitlab
sudo gpasswd -a WeiyiGeek-gitlab docker

# 2. Switch to WeiyiGeek with low permissions-gitlab users pull the latest community version of gitlab
WeiyiGeek-gitlab@gitlab-WeiyiGeek:~$ docker pull gitlab/gitlab-ce
# Using default tag: latest
# latest: Pulling from gitlab/gitlab-ce
# 4 f53fa4d2cf0: Pull complete
# 6 af7c939e38e: Pull complete
# 903 d0ffd64f6: Pull complete
# 04 feeed388b7: Pull complete
# 25 d5e5c7360d: Pull complete
# 0 cc025692f2b: Pull complete
# eac308723fda: Pull complete
# 2135 ce2185ba: Pull complete
# 3 d2db784a8b0: Pull complete
# d2942fac7230: Pull complete
# Digest: sha256:b7daf0c109a62e776f5f72b728a01191059a51f43b5df82c53ef997e877a784f
# Status: Downloaded newer image for gitlab/gitlab-ce:latest
# docker.io/gitlab/gitlab-ce:latest

# 3. Set the persistent directory and run gitlab-server
$ mkdir -vp ~/{config,logs,data}
# mkdir: created directory '/home/WeiyiGeek-gitlab/config'
# mkdir: created directory '/home/WeiyiGeek-gitlab/logs'
# mkdir: created directory '/home/WeiyiGeek-gitlab/data'
$ docker run -d -p 443:443-p 80:80-p 22:22--name gitlab-server --restart always -v /home/WeiyiGeek-gitlab/config:/etc/gitlab -v /home/WeiyiGeek-gitlab/logs:/var/log/gitlab -v /home/WeiyiGeek-gitlab/data:/var/opt/gitlab gitlab/gitlab-ce
512 c45077bafaf1c617ddd6e43f4b8a9a147ca3c8b9e8e889b14d58f151647fe

# 4. Use docker logs to see if gitlab initialization is complete
WeiyiGeek-gitlab@gitlab-WeiyiGeek:~$ docker logs -f --tail 100 512c
==> /var/log/gitlab/puma/puma_stdout.log <=={"timestamp":"2020-10-10T06:14:37.385Z","pid":767,"message":"PumaWorkerKiller: Consuming 3219.65625 mb with master and 4 workers."}

# 5. Host name configuration
sudo tee /etc/hosts <<'EOF'192.168.10.222 gitlab.weiyigeek.top
EOF

PS: How to configure the configuration file address /etc/WeiyiGeek-gitlab/config, please refer to the following article, no more description here;

# Modified key-value pair
sudo egrep -v "^#|^$"/home/WeiyiGeek-gitlab/config/gitlab.rb
[ sudo] password for WeiyiGeek:

external_url 'http://gitlab.WeiyiGeek.top'
user['git_user_name']="Gitlab"
user['git_user_email']="[email protected]"、
gitlab_rails['gitlab_ssh_host']='gitlab.WeiyiGeek.top'
gitlab_rails['gitlab_shell_ssh_port']=2222
gitlab_rails['gitlab_shell_git_timeout']=800
gitlab_rails['time_zone']='Asia/Shanghai'
gitlab_rails['gitlab_email_enabled']=true
gitlab_rails['gitlab_email_from']='[email protected]'
gitlab_rails['gitlab_email_display_name']='Gitlab'
gitlab_rails['gitlab_email_reply_to']='[email protected]'
gitlab_rails['gitlab_email_subject_suffix']='WeiyiGeek-IT'
gitlab_rails['object_store']['enabled']=false
gitlab_rails['object_store']['connection']={}
gitlab_rails['object_store']['storage_options']={}
gitlab_rails['object_store']['proxy_download']=false
gitlab_rails['object_store']['objects']['artifacts']['bucket']= nil
gitlab_rails['object_store']['objects']['external_diffs']['bucket']= nil
gitlab_rails['object_store']['objects']['lfs']['bucket']= nil
gitlab_rails['object_store']['objects']['uploads']['bucket']= nil
gitlab_rails['object_store']['objects']['packages']['bucket']= nil
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket']= nil
gitlab_rails['object_store']['objects']['terraform_state']['bucket']= nil
gitlab_rails['smtp_enable']=true
gitlab_rails['smtp_address']="smtp.exmail.qq.com"
gitlab_rails['smtp_port']=465
gitlab_rails['smtp_user_name']="[email protected]"
gitlab_rails['smtp_password']="Hm595Yb"
gitlab_rails['smtp_domain']="WeiyiGeek.com.top"
gitlab_rails['smtp_authentication']="login"
gitlab_rails['smtp_tls']=true

# Restart the gitlab container after modification
$ docker restart gitlab

0 x02 Operating Instructions####

Gitlab related operations and instructions:

# View Gitlab version
cat /opt/gitlab/embedded/service/gitlab-rails/VERSION

/etc/gitlab/gitlab.rb          #gitlab configuration file
/opt/gitlab                    #gitlab program installation directory
/var/opt/gitlab                #gitlab directory data directory(Via gitlab.The submodule configuration file modified by rb is stored)/var/opt/gitlab/git-data       #Store warehouse data
/var/opt/gitlab/backups        #Default backup directory

# The service structure of Gitlab:
gitlab-shell:Used to process Git commands and modify the authorized keys list
gitlab-workhorse:Lightweight reverse proxy server
logrotate:Log file management tool
nginx:Static web server
postgresql:database
redis:Cache database
sidekiq:Used to execute queue tasks in the background(Asynchronous execution)
unicorn:The GitLab Rails application is hosted on this server.
gitlab-ctl command

Basic commands:

# Common commands
gitlab-ctl reconfigure         #Generate configuration and restart service
gitlab-ctl start               #Start all gitlab components:
gitlab-ctl status              #View the current running status of all services in gitlab
gitlab-ctl stop                #Stop gitlab service
gitlab-ctl pg-upgrade          #Upgrade to the latest version of PostgreSQL

# Separate start|stop|Restart a service
gitlab-ctl start nginx            #Start nginx component
gitlab-ctl stop postgresql        #Stop all gitlab postgresql components:
gitlab-ctl restart unicorn        #Restart related data connection services
gitlab-ctl restart sidekiq        #Restart related data connection services
gitlab-ctl restart  gitlab-workhorse #Restart all gitlab gitlab-workhorse component:

# Log management
gitlab-ctl tail         #View logs of all services
gitlab-ctl tail redis   #Check redis logs in real time (the same goes for extending a certain service)
gitlab-rails command

Basic example:

# gitlab reset password
gitlab-rails console 
> u=User.where(id:1).first        #This is for the administrator, you can also use email, etc.
> u.password ='your_password'    #The password has format restrictions, I only know 8 or more digits, otherwise it will fail to save
> u.password_confirmation ='your_password'> u.save
gitlab-rake command

Basic example:

gitlab-rake gitlab:check SANITIZE=true--trace #Check gitlab
gitlab-rake db:migrate #Database relationship upgrade
gitlab-rake cache:clear #Clean up the redis cache

0 x03 basic configuration####

Gitlab initialization settings#####

0) User theme and language settings

Process: Personal avatar in the upper right corner ->settings->Preferences->Theme settings|Syntax highlighting theme|Custom language and regional related settings

**1) Turn off the Gravatar avatar function for **Network Acceleration

Setting -> Gravatar enabled(Expand Expand) -> not Checked Gravatar enabled -> Sava Changes

WeiyiGeek.

2) User management and registration related settings

Description: Whether users register themselves and set password policies and registered email domain names (white|blacklist) and other information, if closed, the homepage will not display registration;

WeiyiGeek.

WeiyiGeek.

3) Code push reminder function

Description: Add a project to send an email to the designated mailbox to remind the notification after the code is submitted. This is still helpful for code review, at least know who pushed the new code, what content has been updated, etc.

The administrator logs in to the gitlab interface Admin area-&gt; Service Templates -&gt; Emails on push, check Active, Recipients fill in the people who need to push the update of this project, and separate multiple spaces or commas;

WeiyiGeek.

4) Set the port of the ssh shell

Used to process Git commands and modify the authorized keys list

gitlab_rails['gitlab_shell_ssh_port']=9022

5) Modify the gitlab repository (repositories) storage directory

Description: After installation, the repository (repositories) should be placed on a large hard disk, so you need to modify the directory corresponding to the repository.

( Note: I first delete all repositories and then do the following operations. I also delete all the files under the user name in the /var/opt/gitlab/git-data/repositories/username directory. Only the directories are deleted. Empty folder left)

# View disk space(You can operate according to the disk you added)
$df -h
/dev/mapper/cl-home   24G  954M   24G    4%/home

# Create a new warehouse directory
$mkdir -p /home/gitlab/git-data

# Configure the data storage location of gitlab as/home directory
$vi /etc/gitlab/gitlab.rb
git_data_dirs({"default"=>{"path"=>"/home/gitlab/git-data"}})

# Reload configuration
$sudo gitlab-ctl reconfigure

# After reloading, we found that there is data stored under the warehouse address we changed
$ls /home/gitlab/git-data/
repositories

6) Create a development team

Click Create a group on the main page after logging in to Gitlab -> enter the relevant group information to create;

Invite members to join the group: Members->Group members->Invite memeber -> select users to join the newly created group and role permissions;

WeiyiGeek.

7) Project creation and initialization

The composition of Gitlab's git address is the same as github (don't tell me you don't know): gitlab address + user/group + custom name

There are three levels of visible types of Gitlab projects.

For example, the operation process of creating a project in the above development group project is as follows:

  1. Enter the development group -> New Project -> set the project name -> create a project

  2. It is recommended to add the SSH key to Gitlab without password push and pull [email protected]:newproject/secopsdev.git:Setting -> SSH;

$ ls ~/.ssh/
id_ed25519  id_ed25519.pub  id_rsa  id_rsa.pub  known_hosts

WeiyiGeek.

  1. Initialization of the project:
# Basic Settings(Prerequisite you need to download git)
git config --global user.name "WeiyiGeek"
git config --global user.email "[email protected]"

# Create a new repository
git clone [email protected]:newproject/secopsdev.git
cd secopsdev
touch README.md
git add README.md
git commit -m "add README"

# Push existing folder
cd existing_folder
git init
git remote add origin http://gitlab.weiyigeek.top/newproject/secopsdev.git
git add .
git commit -m "Initial commit"

# Push existing Git repositories
cd existing_repo
git remote rename origin old-origin  #Rename the local branch
git remote add origin http://gitlab.weiyigeek.top/newproject/secopsdev.git
  1. It should be noted that the default cannot be submitted. This is caused by the protection of the Master branch. If you want to directly make changes to the master branch but do not want to create a sub-branch, you can use the following methods, provided that the user of Maintainers must upload the Master branch After setting;

Project -> Settings -> Repository -> Brach (Branch permission settings) or delete protection;

WeiyiGeek.

8) User project permission control

Rights management understanding:

Project group & x project authority management:

9) Broadcast information settings

Description: The broadcast information will be displayed to all users. It can be used to notify users of regular system maintenance, upcoming planned upgrades and other information. It will be displayed on the user login interface and when the user submits the code.

Area Admin -> Messages -> Broadcast Information (just set it)

WeiyiGeek.

Gitlab data backup and recovery

Description: GitLab is a version management system of the company's project code. The data is very important, so it must be backed up.

Gitlab provides two kinds of backups:

Local backup

#1. The default directory for GitLab backup is
/var/opt/gitlab/backups 

#2. Modify the backup directory
$vim /etc/gitlab/gitlab.rb
gitlab_rails['backup_path']='/home/backups'
# Backup retention for 7 days
gitlab_rails['backup_keep_time']=604800

#3. Reload the configuration after modifying the configuration
$gitlab-ctl reconfigure

#4. Execute backup command
$gitlab-rake gitlab:backup:create 
# 2020- 04- 1609:44:52+0800- - Dumping database ...
# Dumping PostgreSQL database gitlabhq_production ...[DONE]
# 2020- 04- 1609:44:53+0800- - done
# 2020- 04- 1609:44:53+0800- - Dumping repositories ...
# 2020- 04- 1609:44:53+0800- - done
# 2020- 04- 1609:44:53+0800- - Dumping uploads ...
# 2020- 04- 1609:44:53+0800- - done
# 2020- 04- 1609:44:53+0800- - Dumping builds ...
# 2020- 04- 1609:44:53+0800- - done
# 2020- 04- 1609:44:53+0800- - Dumping artifacts ...
# 2020- 04- 1609:44:53+0800- - done
# 2020- 04- 1609:44:53+0800- - Dumping pages ...
# 2020- 04- 1609:44:53+0800- - done
# 2020- 04- 1609:44:53+0800- - Dumping lfs objects ...
# 2020- 04- 1609:44:53+0800- - done
# 2020- 04- 1609:44:53+0800- - Dumping container registry images ...
# 2020- 04- 1609:44:53+0800- - [ DISABLED]
# Creating backup archive: 1587001493_2020_04_16_12.9.2_gitlab_backup.tar ... done
# Uploading backup archive to remote storage  ... skipped

#5. View backup files,The compressed package includes the entire complete part of GitLab,Approximate format xxxxxxxx_gitlab_backup.tar
$ls /var/opt/gitlab/backups
1587001493_2020_04_16_12.9.2_ gitlab_backup.tar

Supplementary note:

Restart service

systemctl restart crond

Remote backup

Description: In fact, it is to add the pub public key of the backup server on the gitlab server, and then the backup server accesses the Gitlab server for pulling. Of course, you can also use rsync for backup;

# Suppose the environment is as follows:
# Backup server:192.168.1.2
# GitLab server:192.168.1.250

#1. Backup server creation authorized_keys file
mkdir -p /root/.ssh
touch /root/.ssh/authorized_keys 
chmod 400/root/.ssh/authorized_keys

#2. The Gitlab server generates the rsa public key and uploads the certificate to the backup server (the least-privileged user is recommended in the actual environment)
$ssh-keygen -t rsa -C "[email protected]"
# Enter file in which to save the key(/root/.ssh/id_rsa):Error path
# Enter passphrase(empty for no passphrase):Public key password
# Enter same passphrase again:verify password
# Your public key has been saved in/root/.ssh/id_rsa.pub. #save route
# The key fingerprint is:
# SHA256:YahioDs7XZ9No********RS9jWX1NNXo04i4ZHOLY [email protected]
# The key's randomart image is:
# +- - - [ RSA 2048]----+
# |..++|
# |.+. o o.+|
# |. o.oo=.+.|
# |....=. E.++|
# | o o..+ oS=..|
# |.=+.+ o ..|
# |+*..=+|
# | oo= o o .|
# |. o.+|
# +- - - - [ SHA256]-----+

$ls /root/.ssh/
id_rsa  id_rsa.pub

$scp /root/.ssh/id_rsa.pub [email protected]:/root/.ssh/

#3. Backup server will id_rsa.pub file content is appended to authorized_keys file, and modify authorized_The permissions of the keys file
$cat /root/.ssh/id_rsa.pub >>/root/.ssh/authorized_keys

#4. gitlab backup script(Main executive authority appeal)Copy the backup file to the server
$cat >/home/gitlab/auto_backup_to_remote.sh<<'END'
DATE=`date +"%Y_%m_%d"`
BackupIP=192.168.1.2
BackupUser=root
BackupPATH=/var/opt/gitlab/backups

# Delete the local backup 7 days ago
find ${BackupPATH}-type f -mtime +7-name '*.tar'-exec rm {} \;

# If you do not need to back up files, you can also back up the warehouse directory
find ${BackupPATH}-name *${DATE}*-exec scp -r {} ${root}@${BackupIP}:/tmp/ \;
END

$chmod +x /home/gitlab/auto_backup_to_remote.sh

#5. Add time schedule(Execute at 1 am every day), Restart crond service
cat >>/etc/crontab <<END
01***/home/gitlab/auto_backup_to_remote.sh
END

systemctl restart crond

Gitlab data recovery

For example: above we have to backup: /var/opt/gitlab/backups/1587001493_2020_04_16_12.9.2_gitlab_backup.tar;

The operation process is as follows:

# Stop related data connection services (to ensure that there is no new connection to the database and there will be no write data)
gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq

# Specify the recovery file will automatically go to the backup directory to find the format:1587001493_2020_04_16_12.9.2
cd /var/opt/gitlab/backups/&& chmod -R 777*
gitlab-rake gitlab:backup:restore BACKUP=1587001493_2020_04_16_12.9.2

# Start Gitlab
gitlab-ctl restart
Gitlab add email address#####

Email password recovery:

  1. Because gitlab is running all the time, it is necessary to record and back up the configuration file before modification;
cp /etc/gitlab/gitlab.rb{,.bak}
  1. Modify the configuration file: /etc/gitlab/gitlab.rb Here use the company's corporate mailbox to send emails;
$egrep -v "^$|^#"/etc/gitlab/gitlab.rb 
external_url 'http://gitlab.weiyigeek.top'
gitlab_rails['time_zone']='Asia/Shanghai'
gitlab_rails['gitlab_email_enabled']=true
gitlab_rails['gitlab_email_from']='[email protected]'
gitlab_rails['gitlab_email_display_name']='Gitlab CE'
gitlab_rails['gitlab_email_reply_to']='[email protected]'
gitlab_rails['smtp_enable']=true
gitlab_rails['smtp_address']="smtp.exmail.qq.com"
gitlab_rails['smtp_port']=465
gitlab_rails['smtp_user_name']="[email protected]"
gitlab_rails['smtp_password']="email Password"
gitlab_rails['smtp_domain']="weiyigeek.top"
gitlab_rails['smtp_authentication']="login"
gitlab_rails['smtp_enable_starttls_auto']=true
gitlab_rails['smtp_tls']=true
user['git_user_email']="[email protected]"
  1. Save the modification and update the configuration;
# Just refresh the modified configuration file information to the configuration file
gitlab-ctl reconfigure

In fact, a configuration file gitlab.rb is modified above. The configuration in it will be called by this command and written into multiple configuration files to the installation directory: /var/opt/gitlab/gitlab-rails/etc/

$ls /var/opt/gitlab/gitlab-rails/etc/
database.yml         gitlab_shell_secret      gitlab.yml      resque.yml   smtp_settings.rb #You can also directly modify
gitlab_pages_secret  gitlab_workhorse_secret  rack_attack.rb  secrets.yml  unicorn.rb

If you modify the smtp_settings.rb file directly, just restart gitlab-ctl restart.

  1. Console view configuration and test sending
$gitlab-rails console
--------------------------------------------------------------------------------
 GitLab:12.9.2(ac5568eb5d8) FOSS
 GitLab Shell:12.0.0
 PostgreSQL:10.12--------------------------------------------------------------------------------
Loading production environment(Rails 6.0.2)irb(main):001:0> ActionMailer::Base.delivery_method
=>: smtp

irb(main):002:0> ActionMailer::Base.smtp_settings
=>{: authentication=>:login,:address=>"smtp.exmail.qq.com",:port=>465,:user_name=>"[email protected]",:password=>"Pa****20",:domain=>"weiyigeek.top",:enable_starttls_auto=>true,:tls=>true,:ca_file=>"/opt/gitlab/embedded/ssl/certs/cacert.pem"}irb(main):003:0> Notify.test_email('[email protected]','GitLab test mail','GitLab page information').deliver_now
Notify#test_email: processed outbound mail in2.0ms
Delivered mail [email protected](1543.7ms)
Date: Wed,15 Apr 202021:12:37+0800
From: Gitlab CE <[email protected]>
Reply-To: Gitlab CE <[email protected]>

After completing the above operations, go back to the login page and select Forgot your password? Enter the email address of your account to receive the email, and modify the new password according to the email address;

WeiyiGeek.sendemail

Gitlab add LDAP authentication

LDAP configuration instructions: http://gitlab.weiyigeek.top/help/administration/auth/ldap

Step1. Modify the gitlab configuration file to set the parameters for connecting to the LDAP server, that is, the DN in bindPass

vim /etc/gitlab/gitlab.rb
gitlab_rails['ldap_enabled']=true
# gitlab_rails['prevent_ldap_sign_in']=false
###!**remember to close this block with'EOS' below**
gitlab_rails['ldap_servers']= YAML.load <<-'EOS'
 main: # 'main' is the GitLab 'provider ID'ofthis LDAP server
  label:'LDAP'
  host:'10.10.107.245'
  port:389
  timeout:10
  uid:'uid'
  bind_dn:'cn=admin,dc=WeiyiGeek,dc=com,dc=cn'
  password:'WeiyiGeek'
  encryption:'plain' # "start_tls" or "simple_tls" or "plain"
  verify_certificates:false
  smartcard_auth:false
  active_directory:true
  allow_username_or_email_login:true
  lowercase_usernames:false
  block_auto_created_users:false
  base:'dc=WeiyiGeek,dc=com,dc=cn'
  # user_filter:'(&(objectclass=inetOrgPerson)(memberof=cn=gitlab,ou=Group,dc=WeiyiGeek,dc=com,dc=cn))'
  attribute:
  username:['uid']
  email:['mail']
  name:'cn'
  first_name:['sn']
EOS

Step2. LDAP establishes groupOfUniqueNames attribute to obtain user groups and bind users;

dn: cn=gitlab,ou=Group,dc=WeiyiGeek,dc=com,dc=cn
uniqueMember: uid=gituser,ou=People,dc=WeiyiGeek,dc=com,dc=cn

dn: uid=gituser,ou=People,dc=WeiyiGeek,dc=com,dc=cn
objectClass: posixAccount
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
loginShell:/bin/bash
homeDirectory:/home/gituser
userPassword:: e1NTSEF9UGVyM21xc1dJcnV3K1d2bWRiVmVpd3RWZHVVeVN6Tks=
uid: gituser
cn: gituser
uidNumber:10000
gidNumber:10000
sn: gituser
mail: [email protected]

Step3. Login test if the LDAP user does not exist or the filter entry does not exist, an error will be reported Could not authenticate you from Ldapmain because &quot;Invalid credentials for jenkuser&quot;.

WeiyiGeek.

Step4. The login success interface is as follows:

WeiyiGeek.

0 x04 Into the pit solution

(0) Common mistakes of user pull and push code to gitlab

(1) Forgotten gitlab password reset method

u=User.where(id:1).first #This is for the administrator, you can also use email, etc.
u.password ='your_password' #Password has format restrictions, I only know that more than 8 digits, otherwise it will fail to save
u.password_confirmation = 'your_password'
u.save

It is assumed here that the service cannot be linked to the external network (in order to reproduce the following operations);

# (1) View the mail queue that cannot be sent
$postqueue -p
- Queue ID---Size------Arrival Time-----Sender/Recipient-------
# Because there is no such address, it cannot be sent out and can only stay in the mail queue.
908 DCCAC62     6871 Wed Apr 1519:54:32  [email protected] #(Actually, we retrieve the password email)(Host or domain name not found. Name service error for name=example.com type=MX: Host not found,try again)
[email protected]

# (2) Mail queue storage location
$postconf -d | grep queue_directory  #There is no modification by default
queue_directory =/var/spool/postfix

# (3) Go to the directory and find the queue ID number of the mail above
$find /var/spool/postfix/-name 908DCCAC62
/var/spool/postfix/defer/9/908DCCAC62
/var/spool/postfix/deferred/9/908DCCAC62

# (4) Use the tool that comes with postfix to view the content of the email and find the place to change the password
$postcat /var/spool/postfix/deferred/9/908DCCAC62
*** ENVELOPE RECORDS /var/spool/postfix/deferred/9/908DCCAC62 ***
message_size:68712741068710
message_arrival_time: Wed Apr 1519:54:322020
create_time: Wed Apr 1519:54:322020
named_attribute: rewrite_context=local
sender: [email protected]
named_attribute: dsn_orig_rcpt=rfc822;[email protected]
original_recipient: [email protected]
.....
If you did not perform this request, you can safely ignore this email.
Otherwise, click the link below to complete the process:
# Find the following password to modify the connection, and copy it out to the browser to reset the password
http://gitlab.weiyigeek.top/users/password/edit?reset_password_token=gd9MyL7FaSt5R2F3_qA_

WeiyiGeek.

(2) 502 error caused by GitLab database and its solution

Description: Open GitLab's homepage address http:/gitlab.weiyigeek.top/ error 502, restart or check the status and find that it is normal sudo gitlab-ctl status, and check that the port number is not occupied, check the refresh configuration gitlab- ctl reconfigure found an error;

The error message is as follows:

# gitlab-ctl reconfigure 
Running handlers:
There was an error running gitlab-ctl reconfigure:

bash[migrate gitlab-rails database](gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with[0], but received '1'---- Begin output of"bash""/tmp/chef-script20190308-65247-12ck9rp"----
STDOUT: rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
 Is the server running locally and accepting
 connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:52:in`block (3 levels) in <top (required)>
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => gitlab:db:configure(See full trace by running task with--trace)
STDERR:---- End output of"bash""/tmp/chef-script20190308-65247-12ck9rp"----
Ran "bash""/tmp/chef-script20190308-65247-12ck9rp" returned 1

# gitlab-ctl tail

WeiyiGeek.

(3) gitlab-ce version upgrade record

In order to ensure data security, we will upgrade slowly step by step. Using official sources may be slower. You can use domestic sources (using Omnibus for upgrades and direct rpm package deployment): https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum

Upgrade method: first upgrade to the last version of the current major version (here is my suggestion, generally the last few versions are fine), and then upgrade across major versions, first upgrade from the lower version of the major version, and then to the end Version upgrade, and so on; because I tried to upgrade directly across too many versions, and there were exceptions. In fact, it was mainly the data import and backup problem of the database. If the new version is too much related to the old version, an error will be reported.

For example, my upgrade process is very long and I also did a simple test

# upgraded version
8.3.1-->8.3.7-->8.8.5-->8.17.4-->9.5.6-->10.0.6-->10.8.5-->11.0.3

# Out of service
gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq
gitlab-ctl stop nginx

# Create data backup to prevent update failure(Very important)
gitlab-rake gitlab:backup:create

# Install the upgrade package
# yum update gitlab-ce #Upgrade GitLab-ce version(This method is not recommended)
rpm -Uvh gitlab-7.4.2_omnibus-1.el6.x86_64.rpm 

# Reload configuration&Restart Gitlab service
gitlab-ctl reconfigure
gitlab-ctl restart

For example, whether there is any abnormality in the data backed up by the project, user information, response speed, etc., I personally think that it is relatively reliable.

In fact, as long as we have upgraded before, we won't be like this anymore. We upgraded from such a low version all at once. It is generally recommended to keep the same version number. Too new may have bugs, and too old may have many historical problems.

After the upgrade is complete, you will find several more processes. It may have higher requirements for system hardware, such as memory requirements.

Suggestion: If you simply want a code repository for non-essential functional requirements, it would be good to use it personally between 11.x.-12.3.x.

**(4) gitlab-rails database failed to initialize **

Environmental description

Centos7:3.10.0-327.el7.x86_64
gitlab version: the original is 8.8.Version 5 is now upgraded to a newer version;

Problem: After uninstalling the old version of gitlab on the same Centos7 machine, and then reinstalling the new version of gitlab-ce, the configuration was modified after the installation, and the following error occurred when initializing the configuration:

Running handlers:
There was an error running gitlab-ctl reconfigure:

bash[migrate gitlab-rails database](gitlab::database_migrations line 49) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with[0], but received '1'---- Begin output of"bash""/tmp/chef-script20190628-7065-vx17en"----
STDOUT: rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
 Is the server running locally and accepting
 connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:49:in`block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => gitlab:db:configure(See full trace by running task with--trace)
STDERR:---- End output of"bash""/tmp/chef-script20190628-7065-vx17en"----
Ran "bash""/tmp/chef-script20190628-7065-vx17en" returned 1

Reason: Because the old data was not deleted when the old version was uninstalled, when installing directly, the program detects the configuration file, and the data file will be skipped without overwriting. The new version uses the data file of the old version because the version is too different It will be unable to identify the cause.

Solution: After uninstalling the old version, remember to delete the old installation data configuration file, installation directory, data directory, etc, and reinstall the new version of gitlab-ce to start normally.

rm -rf /var/opt/gitlab//opt/gitlab//etc/gitlab/

(5) Gitlab database postgresql updates account information

Problem description: Because the internal Ldap authentication server hard disk is broken, the Ldap service running on it cannot perform centralized authentication on the Jenkins and Gitlab platforms. As a result, the account on Gitlab cannot log in to the platform, so it cannot submit the pull Code too.

The process of selecting method 2 for operation is as follows:

#2. Modify the postgresql configuration file to change listen_addresses to *
$vim /var/opt/gitlab/postgresql/data/postgresql.conf

#------------------------------------------------------------------------------

CONNECTIONS AND AUTHENTICATION

#------------------------------------------------------------------------------

- Connection Settings -

listen_addresses = '*' # what IP address(es) to listen on;

comma-separated list of addresses;

defaults to 'localhost', '*' = all

( change requires restart)

port = 5432 # (change requires restart)
max_connections = 200 # (change requires restart)

Note: Increasing max_connections costs ~400 bytes of shared memory per

connection slot, plus lock space (see max_locks_per_transaction).

superuser_reserved_connections = 3 # (change requires restart)

unix_socket_directories = '/var/opt/gitlab/postgresql' # (change requires restart)

#3. Restart postgresql service
gitlab-ctl restart postgresql

#4. Use Navicat tool to connect to postgresql to connect to the database. The initial database gitlabhq_production and user name below are the default, and the password is blank.

After connecting, find the corresponding identities table and modify the corresponding field. In fact, it is to modify the user_id in the field. What I do here is to change the user_id of the user to a negative number, so that it will not be found during Ldap authentication, so standard authentication (This is done without restarting Gitlab).

WeiyiGeek.

Supplementary note:

0 x05 Script Appendix####

GitLab one-click installation configuration script

#! /bin/bash
# Desc:Gitlab code server automated deployment
# Author:WeiyiGeek
# SupportOS:CentOS7 / CentOS8

GITLAB_BASEDOMAIN=weiyigeek.top
GITLAB_VERSION=12.9.2
GITlABOS7=gitlab-ce-${GITLAB_VERSION}-ce.0.el7.x86_64.rpm
GITlABOS8=gitlab-ce-${GITLAB_VERSION}-ce.0.el8.x86_64.rpm

GITLABRUNNER_VERSION=12.9.0-1
GITLABRUNNER_NAME=gitlab-runner-${GITLABRUNNER_VERSION}.x86_64.rpm	
CheckOSVersion=$(uname -r | grep -c el8)

## [ Mirror source settings]functionrepoChange(){
 mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.$(date +"%Y%m%d").backup
 if[ $CheckOSVersion -eq 1];then
 # CentOS8 source
 curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-8.repo
 # Install the epel configuration package and replace the address with the address of the Aliyun mirror station
 dnf install -y https://mirrors.aliyun.com/epel/epel-release-latest-8.noarch.rpm
 sed -i 's|^#baseurl=https://download.fedoraproject.org/pub|baseurl=https://mirrors.aliyun.com|'/etc/yum.repos.d/epel*
 sed -i 's|^metalink|#metalink|'/etc/yum.repos.d/epel*
 dnf clean all
 dnf makecache
 else
 # CentOS7 source
 curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
 sed -i -e '/mirrors.cloud.aliyuncs.com/d'-e '/mirrors.aliyuncs.com/d'/etc/yum.repos.d/CentOS-Base.repo
 wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
# Add to trust the GPG public key in GitLab
sudo cat >/etc/yum.repos.d/gitlab-ce.repo <<EOF
[ gitlab-ce]
name=Gitlab CE Repository
baseurl=https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el7/
gpgcheck=0
enabled=1
gpgkey=https://packages.gitlab.com/gpg.key
EOF
 yum clean all
 yum makecache
 fi
}

#[ yum installation]functionyumInstall(){
 # View available versions neng'b
 yum list gitlab-ce --showduplicates
 # Install the latest version by default
 yum install -y gitlab-ce
 # Install specified version 12.3.5
 # yum install gitlab-ce-12.3.5-ce.0.el7.x86_64.rpm
}

#[ rpm installation-Recommended way]functionOmnibusInstall(){if[ $CheckOSVersion -eq 1];then
 wget -O $GITlABOS8 https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el8/${GITlABOS8}
 rpm -i $GITlABOS8
 else
 wget -O $GITlABOS7 https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el7/${GITlABOS7}
 rpm -i  $GITlABOS7
 fi
} functiongitlabSetting(){
 sed -i "s#example.com#${GITLAB_BASEDOMAIN}#g"/etc/gitlab/gitlab.rb
 echo "127.0.0.1 gitlab.${GITLAB_BASEDOMAIN}">/etc/hosts
} functionuseage(){
 echo -e "\e[32m# Description:Gitlab automated installation and deployment script"
 echo -e "usage: $0 [rpm|yum] #Specify rpm installation or yum installation"
 echo -e "Author:WeiyiGeek\e[0m"}

#[ Less than 12.3.Set only for x version]functionChinesization(){
 # Stop gitlab
 gitlab-ctl stop

 # Get the currently installed version patch
 git clone https://gitlab.com/xhang/gitlab.git
 cd gitlab
 gitlab_version=$(cat /opt/gitlab/embedded/service/gitlab-rails/VERSION)

 # Generate the corresponding version patch file
 git diff remotes/origin/12-3-stable remotes/origin/12-3-stable-zh >../${gitlab_version}-zh.diff

 # When patching, it will prompt that some patch files do not exist. Be sure to skip these files. Otherwise, you will get an error when you reconfig.
 patch -d /opt/gitlab/embedded/service/gitlab-rails -p1 <../${gitlab_version}-zh.diff
 gitlab-ctl reconfigure
 gitlab-ctl restart
}

## [ Installation configuration script entry function]functionmain(){
 # Turn off Sellinux
 echo "Current Selinux: $(getenforce)"
 setenforce 0
 sed -i 's/SELINUX=enforcing/SELINUX=disabled/g'/etc/selinux/config
 echo "Set up Selinux: $(getenforce)"

 # Environment dependent installation
 repoChange
 sudo yum install -y curl policycoreutils openssh-server wget postfix git htop ncdu net-tools
 systemctl enable postfix
 systemctl start postfix

 # Protective wall setting
 sudo firewall-cmd --permanent --add-service=http
 sudo firewall-cmd --permanent --add-service=https
 sudo systemctl reload firewalld

 # Choose installation method
 if[ $1=="rpm"];then
 OmnibusInstall
 elif [ $1=="yum"];then 
 yumInstall
 else
 usage
 fi

 gitlabSetting
 gitlab-ctl reconfigure
 gitlab-ctl start
 gitlab-ctl status
}

#[ Parameter verification]if[ $# -ne 1];then
 usage
else 
 main $1
fi

Original blog address: https://weiyigeek.top/2019/7/98.html

**WeiyiGeek Blog-In order to be far away, every step under your feet must be done. **

The source of this article is the Blog site (Friend chain exchange, please email me):

Recommended Posts

Enterprise self-built GitLab code warehouse installation and basic configuration use
GitLab installation and basic use
Ubuntu16.04 configuration OpenCV3.4.2 and basic use
Basic configuration and common software installation after ubuntu 18.04 installation
Installation, configuration and uninstallation of GitLab in Ubuntu19.1
OpenMPI-Ubuntu installation and configuration
Centos mysql installation and configuration
Centos7 installation and configuration prometheus
CentOS 7 installation and configuration PPTP
Ubuntu16.04 installation and simple configuration
CentOS installation and configuration cmake
Centos7.5 installation and configuration MongoDB4.0.4
CentOS 7 installation and configuration PPTP
centos7 kvm installation and use
CentOS7 postgresql installation and use
Ubuntu PostgreSQL installation and configuration
Centos7 elk7.1.1 installation and use
Ubuntu basic settings: introduction to the installation and use of openssh-server
Centos7 installation and configuration of Jenkins
CentOS 8 install Git and basic configuration
Centos7 hadoop cluster installation and configuration
CentOS7 installation and maintenance of Gitlab
Java-JDK installation and configuration under CentOS
CentOS 7 Tomcat service installation and configuration
Ubuntu 19.1 installation and configuration Chinese environment
CentOS NTP server installation and configuration
Configuration and beautification after Ubuntu installation (1)
Nginx installation and configuration load (ubuntu12.04)
Ubuntu configuration source and installation software
Centos7 mysql database installation and configuration
CentOS 7 system installation and configuration graphic tutorial
Installation and use of Mysql under CentOS
Tomcat installation and configuration under CentOS 7 (Tomcat startup)
MySQL 8.0 installation, deployment and configuration under CentOS 6/7
Installation and configuration of redis under centos7
Centos7 installation and deployment of gitlab server
Ubuntu18.04 Server version installation and use (graphic)
Installation and use of Win10 subsystem Ubuntu
Centos7 hive stand-alone mode installation and configuration
KVM installation and preliminary use under CentOS 7.2
DLNA/UPnP Server installation and configuration under Ubuntu 12.04