파일시스템 양방향 동기화하기 - rsync, lsyncd
2024.01.03 10:24
원문 : http://www.ischo.net -- 조인상 // 시스템 엔지니어
Writer : http://www.ischo.net -- ischo // System Engineer in Replubic Of Korea
+++++++++++++++++++++++++++++++++++++++++++++++++++++++
1. 시나리오
- 다음 2개의 서버의 디렉토리내 파일들을 양방향으로 동기화
Server1 : /data/files (소유권 계정 dsnw)
Server2 : /data/files (소유권 계정 dsnw)
- 운영환경
OS : Rocky Linux 9.1
ssh port : TCP 2142
2. rsync를 이용하여 동기화하기
2-1. ssh key 생성
- Server1 측 key생성
[server1]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa
Your public key has been saved in /root/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:ukSjPTTh+T8/GapLPH/DJ81DxZpZtHOXkK/4o1pGytk root@server1
The key's randomart image is:
+---[RSA 3072]----+
| . |
| o .|
| . ooo|
| . o =*|
| B S .. .*+|
| = *. =...= |
| . = =+ E.* |
| . + += Bo= |
| . o++=o=.. |
+----[SHA256]-----+
- Server2 측 key생성
[server2]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa
Your public key has been saved in /root/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:ukSjPTTh+T8/GapLPH/DJ81DxZpZtHOXkK/4o1pGytk root@server2
The key's randomart image is:
+---[RSA 3072]----+
| . |
| o .|
| . ooo|
| . o =*|
| B S .. .*+|
| = *. =...= |
| . = =+ E.* |
| . + += Bo= |
| . o++=o=.. |
+----[SHA256]-----+
2-2. 생성된 ssh-key 를 반대편 서버로 전송
- Server1 측 key를 Server2로 복사
[server1]# ssh-copy-id -i ~/.ssh/id_rsa.pub -p 2142 dsnw@server1
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '[server2]:2142 ([10.10.10.31]:2142)' can't be established.
ED25519 key fingerprint is SHA256:SMxb/Xo++ppBX6XMQbFFRUlZ7f2Z33wd/7T/p6RjzIw.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
dsnw@server2's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh -p '2142' 'dsnw@server2'"
and check to make sure that only the key(s) you wanted were added.
- Server2 측 key를 Server1로 복사
[server2]# ssh-copy-id -i ~/.ssh/id_rsa.pub -p 2142 dsnw@server2
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '[server1]:2142 ([10.10.10.30]:2142)' can't be established.
ED25519 key fingerprint is SHA256:SMxb/Xo++ppBX6XMQbFFRUlZ7f2Z33wd/7T/p6RjzIw.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
dsnw@server2's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh -p '2142' 'dsnw@server1'"
and check to make sure that only the key(s) you wanted were added.
2-3. 접속테스트
- Server1 에서 Server2로 패스워드 없이 접속
[server1]# ssh -p 2142 dsnw@server2
- Server2 에서 Server1로 패스워드 없이 접속
[server2]# ssh -p 2142 dsnw@server1
2-4. rsync 명령어로 데이터 동기화
- Server1 에서 Server2로 데이터 동기화
[server1]# rsync -Pcarvz -e 'ssh -p 2142' --progress --delete /data/files/ dsnw@server2:/data/files/
- Server2 에서 Server1로 데이터 동기화
[server1]# rsync -Pcarvz -e 'ssh -p 2142' --progress --delete dsnw@server2:/data/files/ /data/files/
2-5. 스크립트로 만들기
[server1]# vi /data/script/rsync_a_miniute.sh
rsync -Pcarvz -e 'ssh -p 2142' --progress --delete /data/files/ dsnw@server2:/data/files/
sleep 1
rsync -Pcarvz -e 'ssh -p 2142' --progress --delete dsnw@server2:/data/files/ /data/files/
:wq!
2-6. cron에 등록
*/1 * * * * /data/script/rsync_a_miniute.sh
2-7. 이 방식의 문제점
- 실시간 동기화가 아니다.
- 한쪽 방향 동기화의 경우 문제가 없지만, 양방향 동기화의 경우 조건에 따라 동기화가 의도와 다르게 동작할 수 있다.
예) Server1에서 File1이 삭제되고 Server2에서 File2가 생성된 경우, 동기화 결과로 Server1에서 삭제된 File1이 다시 생성될 수 있다. 역순의 경우도 동일하다.
3. lsyncd를 이용한 양방향 동기화
- rsync와는 다르게 데몬 형태로 동작하는 실시간 동기화이다.
- rsync와는 다르게 양쪽 서버간 변경내용을 오류없이 확인하여 동기화해줄 수 있다.
3-1. lsyncd 설치
# dnf install -y epel-release
# dnf install lsyncd
# systemctl enable lsyncd
3-2. lsyncd conf 설정
[server1]# vi /etc/lsyncd.conf
settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd.status",
statusInterval = 20,
maxProcesses = 1
-- nodaemon = true,
-- insist = true
}
sync {
default.rsyncssh,
source = "/data/files",
host = "dsnw@server2",
targetdir = "/data/files",
delete = 'running',
rsync = {
update = true,
times = true,
archive = true,
compress = true,
perms = true,
acls = true,
owner = true
},
ssh = {
port = 2142
}
}
[server2]# vi /etc/lsyncd.conf
settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd.status",
statusInterval = 20,
maxProcesses = 1
-- nodaemon = true,
-- insist = true
}
sync {
default.rsyncssh,
source = "/data/files",
host = "dsnw@server1",
targetdir = "/data/files",
delete = 'running',
rsync = {
update = true,
times = true,
archive = true,
compress = true,
perms = true,
acls = true,
owner = true
},
ssh = {
port = 2142
}
}
3-3. 데몬 시작
[server1]# service lsyncd start
[server1]# ps -ef | grep lsync
root 51813 1 0 1월02 ? 00:00:00 /usr/bin/lsyncd -nodaemon /etc/lsyncd.conf
[server2]# service lsyncd start
[server2]# ps -ef | grep lsync
root 51813 1 0 1월02 ? 00:00:00 /usr/bin/lsyncd -nodaemon /etc/lsyncd.conf
3-4. 커널파라미터 수정
[server1]# echo "fs.inotify.max_queued_events = 16384" >> /etc/sysctl.conf
[server1]# echo "fs.inotify.max_user_instances = 128 " >> /etc/sysctl.conf
[server1]# echo "fs.inotify.max_user_watches = 500000" >> /etc/sysctl.conf
[server1]# sysctl -p
[server2]# echo "fs.inotify.max_queued_events = 16384" >> /etc/sysctl.conf
[server2]# echo "fs.inotify.max_user_instances = 128 " >> /etc/sysctl.conf
[server2]# echo "fs.inotify.max_user_watches = 500000" >> /etc/sysctl.conf
[server2]# sysctl -p
3-5. 테스트
양쪽 서버 파일들을 생성, 삭제하면서 동기화 여부를 점검한다.