CENTOS7 manually install CEPH original

Deploy mon

1. Configure your hostname
   Don’t engage in weird hostnames, weird hostnames may have weird problems, just be honest node1, this kind of letters plus numbers
   Centos7 modify host: hostnamectl set-hostname node1

2. Ready to refer to the configuration file
   [ global]
   auth service required = cephx
   filestore xattr use omap = true
   auth client required = cephx
   auth cluster required = cephx
   mon host = 192.168.0.1,192.168.0.2,192.168.0.3
   mon initial members = node1,node2,node3
   fsid = 87619a71-34dc-4203-8c52-995c374562a6
   [ mon.node1]
   host = node1
   mon addr = 192.168.0.1:6789
   [ mon.node2]
   host = node2
   mon addr = 192.168.0.2:6789
   [ mon.node3]
   host = node3
   mon addr = 192.168.0.3:6789
   Write to /etc/ceph/ceph.conf

3. Clean up the environment
   rm -fr /var/lib/ceph/*
   rm -fr /tmp/monmap /tmp/ceph.mon.keyring

4. Make a new cluster uuid
   [ root@node1 /data]# uuidgen
   87619 a71-34dc-4203-8c52-995c374562a6
   Modify your ceph.conf "fsid = a7f64266-0894-4f1e-a635-d0aecca0e993"

5. Determine your mon collection
   You have to do several mons and write several in your configuration file, usually singular, such as 1 mon and 3 mon
   mon initial members = node1,node2,node3
   mon host = 192.168.0.1,192.168.0.2,192.168.0.3

6. Make a key ring
   Do mon key
   ceph-authtool –create-keyring /tmp/ceph.mon.keyring –gen-key -n mon. –cap mon ‘allow *’
   Make the administrator key and merge it with the mon key
   ceph-authtool –create-keyring /etc/ceph/ceph.client.admin.keyring –gen-key -n client.admin –set-uid=0 –cap mon ‘allow *’ –cap osd ‘allow *’ –cap mds ‘allow’
   ceph-authtool /tmp/ceph.mon.keyring –import-keyring /etc/ceph/ceph.client.admin.keyring

7. Do a good job of mon view, this is very important, every node of mon must be entered
   monmaptool –create –add node1 192.168.0.1 –add node2 192.168.0.2 –add node3 192.168.0.3 –fsid 87619a71-34dc-4203-8c52-995c374562a6 /tmp/monmap

8. Make the mon data directory
   mkdir -p /var/lib/ceph/mon/ceph-node1

9. Initialize and create mon file system
   ceph-mon –mkfs -i node1 –monmap /tmp/monmap –keyring /tmp/ceph.mon.keyring
   touch /var/lib/ceph/mon/ceph-node1/done #This must be necessary, the logo has been prepared ok

10. Start service
    /etc/init.d/ceph start mon.node1
    /etc/init.d/ceph start mon.node2
    /etc/init.d/ceph start mon.node3

#! /bin/sh
for disk in$(ls /dev/sd*1);do
  # Skipped disk device
  if["$disk"="/dev/sda1"-o "$disk"="/dev/sdb1"];then
    echo "skip $disk"else
    i=$(ceph osd create)
    echo "mkxfs..$disk"
    mkfs.xfs -f $disk
    mkdir -p /var/lib/ceph/osd/ceph-$i
    mount -t xfs -o noatime,inode64 -- $disk /var/lib/ceph/osd/ceph-$i
    ceph-osd -i $i --mkfs --mkkey --osd-uuid b23b48bf-373a-489c-821a-31b60b5b5af0
    ceph auth add osd.$i osd 'allow *' mon 'allow profile osd'-i /var/lib/ceph/osd/ceph-$i/keyring
    ceph osd crush add osd.$i 1.0 host=node1
    echo "[osd.$i]">>/etc/ceph/ceph.conf
    echo "host = node1">>/etc/ceph/ceph.conf
    /etc/init.d/ceph start osd.$i
  fi
done

Deploy mds and start the file system#

1. Add the following content in the configuration file /etc/ceph/ceph.conf

[ mds]
mds data = /var/lib/ceph/mds/mds.id keyring = /var/lib/ceph/mds/mds.id/mds.$id.keyring
[ mds.0]
host = {hostname}

2. Create a directory and key
   mkdir -p /var/lib/ceph/mds/mds.0
   ceph auth get-or-create mds.0 mds ‘allow ‘ osd ‘allow *’ mon ‘allow rwx’ > /var/lib/ceph/mds/mds.0/mds.0.keyring

3. Start mds
   /etc/init.d/ceph start mds.0

4. Detect ceph -s

Do ceph file system

1. Increase metadata storage pool and data storage pool
   $ ceph osd pool create cephfs_data <pg_num>
   $ ceph osd pool create cephfs_metadata <pg_num>
   About pg_num
   it is mandatory to choose the value of pg_num because it cannot be calculated automatically. Here are a few values commonly used:

Less than 5 OSDs set pg_num to 128
Between 5 and 10 OSDs set pg_num to 512
Between 10 and 50 OSDs set pg_num to 4096
If you have more than 50 OSDs, you need to understand the tradeoffs and how to calculate the pg_num value by yourself

( OSDs * 100)
Total PGs = ————
pool size

2. Generate file system
   $ceph osd lspools
   5 cephfs_data,6 cephfs_metadata,
   $ceph mds newfs 6 5 –yes-i-really-mean-it

File system mount

1. File system mounting requires custom configuration file /etc/ceph/ceph.conf
   [ client]
   log file = /data/logs/ceph-client.log
   keyring = /etc/ceph/keyring
   The keyring can be obtained from the ceph auth list
2. Mount
   ceph-fuse -m 192.168.0.1:6789 /mnt/ceph/

Without performance optimization, Ceph will store the logs on the same hard disk as the OSD data. The OSD pursuing high performance can use a separate hard disk to store log data. For example, a solid state hard disk can provide high performance logs.

The default value of osd journal size is 0, so you have to set it in ceph.conf. The log size should be the product of the filestore max sync interval and the expected throughput multiplied by 2.

osd journal size = {2 * (expected throughput * filestore max sync interval)}
The expected throughput should consider the expected hard disk throughput (that is, the continuous data transfer rate) and network throughput. For example, the speed of a 7200 rpm hard disk is roughly 100MB/s. The smaller (min()) of the hard disk and network throughput is a relatively reasonable throughput. Some users start with a log size of 10GB, for example:

osd journal size = 10000