Discussion:
Jenkins Backup and Restore
g***@gmail.com
2018-11-01 20:36:19 UTC
Permalink
We have a Jenkins instance running and we are trying to setup a backup and
restore mechanism in case of any mishaps. We are looking for a full backup
of the Jenkins and so decided to go with PeriodicBackup plugin. With this
plugin, the backup works fine. But, we are trying to restore it in a
completely new Jenkins setup and not able to restore. When we pick the
backup file that we want to restore from, it says restoring, but nothing
happens. Even after waiting out for over 30 mins, nothing is restored. As
a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
(/var/lib/jenkins) folder to 'jenkins_backup' and created another folder
called 'jenkins' in the same location. In this folder, I copied all the
files from the backup and then started Jenkins (NOTE: I didn't change the
.owner file. I'm using the .owner file of the new Jenkins only). Jenkins
came up and I was able to see all the configurations from the old one.
Everything was fine. I let it like that overnight and in the morning when
I came and tried to access it, Jenkins was way too slow and I was not able
to save any new configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI to come
up. Am I doing anything wrong here? Any suggestions for my requirement?

P.S. - I tried SCMSync plugin and ThinBackup. But, problem with these are
that they don't do full backup. But, with ThinBackup both backup and
restore works fine.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Rob Pickerill
2018-11-01 20:41:06 UTC
Permalink
Hi

Have you taken a look through:
https://wiki.jenkins.io/display/JENKINS/I%27m+getting+too+many+open+files+error
and
https://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/
- these give guidance on handling and increasing the open file limit so you
can restore your backup.

Thanks
Post by g***@gmail.com
We have a Jenkins instance running and we are trying to setup a backup and
restore mechanism in case of any mishaps. We are looking for a full backup
of the Jenkins and so decided to go with PeriodicBackup plugin. With this
plugin, the backup works fine. But, we are trying to restore it in a
completely new Jenkins setup and not able to restore. When we pick the
backup file that we want to restore from, it says restoring, but nothing
happens. Even after waiting out for over 30 mins, nothing is restored. As
a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
(/var/lib/jenkins) folder to 'jenkins_backup' and created another folder
called 'jenkins' in the same location. In this folder, I copied all the
files from the backup and then started Jenkins (NOTE: I didn't change the
.owner file. I'm using the .owner file of the new Jenkins only). Jenkins
came up and I was able to see all the configurations from the old one.
Everything was fine. I let it like that overnight and in the morning when
I came and tried to access it, Jenkins was way too slow and I was not able
to save any new configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI to
come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with these are
that they don't do full backup. But, with ThinBackup both backup and
restore works fine.
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com
<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAKotrh3N0yO5H-n7fcGf%3D%3DZJQyh76Qhag_ZcVjkw3mNrFuJafQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Artur Szostak
2018-11-26 21:12:54 UTC
Permalink
Just my opinion: but I would forget about the backup plugins in Jenkins for full production backups. They all do a poor job when it comes to a full backup for a production system that also needs high availability. The best that can be done at the moment is to run Jenkins on a file system or data store that is able to perform copy-on-write snapshots. From within Jenkins you will want to setup a simple job that triggers the copy-on-write snapshot when no other job is running. Then use your normal production backup solutions to backup the snapshot in the background.

Cheers

Artur

________________________________________
From: jenkinsci-***@googlegroups.com <jenkinsci-***@googlegroups.com> on behalf of ***@gmail.com <***@gmail.com>
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore

We have a Jenkins instance running and we are trying to setup a backup and restore mechanism in case of any mishaps. We are looking for a full backup of the Jenkins and so decided to go with PeriodicBackup plugin. With this plugin, the backup works fine. But, we are trying to restore it in a completely new Jenkins setup and not able to restore. When we pick the backup file that we want to restore from, it says restoring, but nothing happens. Even after waiting out for over 30 mins, nothing is restored. As a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME' (/var/lib/jenkins) folder to 'jenkins_backup' and created another folder called 'jenkins' in the same location. In this folder, I copied all the files from the backup and then started Jenkins (NOTE: I didn't change the .owner file. I'm using the .owner file of the new Jenkins only). Jenkins came up and I was able to see all the configurations from the old one. Everything was fine. I let it like that overnight and in the morning when I came and tried to access it, Jenkins was way too slow and I was not able to save any new configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI to come up. Am I doing anything wrong here? Any suggestions for my requirement?

P.S. - I tried SCMSync plugin and ThinBackup. But, problem with these are that they don't do full backup. But, with ThinBackup both backup and restore works fine.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com<mailto:jenkinsci-users+***@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/c318b5c6cfce4024ab5d859fac9fe1f3%40partner.eso.org.
For more options, visit https://groups.google.com/d/optout.
John Mellor
2018-11-26 21:34:55 UTC
Permalink
I have had restore issues with *ALL* of the backup plugins that I have
tried. The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink. This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.

Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).
Post by Artur Szostak
Just my opinion: but I would forget about the backup plugins in
Jenkins for full production backups. They all do a poor job when it
comes to a full backup for a production system that also needs high
availability. The best that can be done at the moment is to run
Jenkins on a file system or data store that is able to perform copy-
on-write snapshots. From within Jenkins you will want to setup a
simple job that triggers the copy-on-write snapshot when no other job
is running. Then use your normal production backup solutions to
backup the snapshot in the background.
Cheers
Artur
________________________________________
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore
We have a Jenkins instance running and we are trying to setup a
backup and restore mechanism in case of any mishaps. We are looking
for a full backup of the Jenkins and so decided to go with
PeriodicBackup plugin. With this plugin, the backup works
fine. But, we are trying to restore it in a completely new Jenkins
setup and not able to restore. When we pick the backup file that we
want to restore from, it says restoring, but nothing happens. Even
after waiting out for over 30 mins, nothing is restored. As a work
around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
(/var/lib/jenkins) folder to 'jenkins_backup' and created another
folder called 'jenkins' in the same location. In this folder, I
I didn't change the .owner file. I'm using the .owner file of the
new Jenkins only). Jenkins came up and I was able to see all the
configurations from the old one. Everything was fine. I let it like
that overnight and in the morning when I came and tried to access it,
Jenkins was way too slow and I was not able to save any new
configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI
to come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with
these are that they don't do full backup. But, with ThinBackup both
backup and restore works fine.
--
You received this message because you are subscribed to the Google
Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it,
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
;.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0e5a17f5dc808937b44f25f634cc6eeb76fb907a.camel%40esentire.com.
For more options, visit https://groups.google.com/d/optout.
g***@gmail.com
2018-11-26 21:41:18 UTC
Permalink
I wish for a tool like Jenkins it had a backup and restore option built in.
Post by John Mellor
I have had restore issues with *ALL* of the backup plugins that I have
tried. The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink. This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.
Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).
Post by Artur Szostak
Just my opinion: but I would forget about the backup plugins in
Jenkins for full production backups. They all do a poor job when it
comes to a full backup for a production system that also needs high
availability. The best that can be done at the moment is to run
Jenkins on a file system or data store that is able to perform copy-
on-write snapshots. From within Jenkins you will want to setup a
simple job that triggers the copy-on-write snapshot when no other job
is running. Then use your normal production backup solutions to
backup the snapshot in the background.
Cheers
Artur
________________________________________
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore
We have a Jenkins instance running and we are trying to setup a
backup and restore mechanism in case of any mishaps. We are looking
for a full backup of the Jenkins and so decided to go with
PeriodicBackup plugin. With this plugin, the backup works
fine. But, we are trying to restore it in a completely new Jenkins
setup and not able to restore. When we pick the backup file that we
want to restore from, it says restoring, but nothing happens. Even
after waiting out for over 30 mins, nothing is restored. As a work
around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
(/var/lib/jenkins) folder to 'jenkins_backup' and created another
folder called 'jenkins' in the same location. In this folder, I
I didn't change the .owner file. I'm using the .owner file of the
new Jenkins only). Jenkins came up and I was able to see all the
configurations from the old one. Everything was fine. I let it like
that overnight and in the morning when I came and tried to access it,
Jenkins was way too slow and I was not able to save any new
configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI
to come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with
these are that they don't do full backup. But, with ThinBackup both
backup and restore works fine.
--
You received this message because you are subscribed to the Google
Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it,
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com
<
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
Post by Artur Szostak
;.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/3501a70d-1b2f-4163-88e8-a2d497f64d43%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
j***@godaddy.com
2018-11-26 22:03:05 UTC
Permalink
The beauty of Jenkins is that it's NOT built-in, but can be easily added as
a plugin (of which there are several).
Post by g***@gmail.com
I wish for a tool like Jenkins it had a backup and restore option built in.
Post by John Mellor
I have had restore issues with *ALL* of the backup plugins that I have
tried. The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink. This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.
Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).
Post by Artur Szostak
Just my opinion: but I would forget about the backup plugins in
Jenkins for full production backups. They all do a poor job when it
comes to a full backup for a production system that also needs high
availability. The best that can be done at the moment is to run
Jenkins on a file system or data store that is able to perform copy-
on-write snapshots. From within Jenkins you will want to setup a
simple job that triggers the copy-on-write snapshot when no other job
is running. Then use your normal production backup solutions to
backup the snapshot in the background.
Cheers
Artur
________________________________________
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore
We have a Jenkins instance running and we are trying to setup a
backup and restore mechanism in case of any mishaps. We are looking
for a full backup of the Jenkins and so decided to go with
PeriodicBackup plugin. With this plugin, the backup works
fine. But, we are trying to restore it in a completely new Jenkins
setup and not able to restore. When we pick the backup file that we
want to restore from, it says restoring, but nothing happens. Even
after waiting out for over 30 mins, nothing is restored. As a work
around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
(/var/lib/jenkins) folder to 'jenkins_backup' and created another
folder called 'jenkins' in the same location. In this folder, I
I didn't change the .owner file. I'm using the .owner file of the
new Jenkins only). Jenkins came up and I was able to see all the
configurations from the old one. Everything was fine. I let it like
that overnight and in the morning when I came and tried to access it,
Jenkins was way too slow and I was not able to save any new
configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI
to come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with
these are that they don't do full backup. But, with ThinBackup both
backup and restore works fine.
--
You received this message because you are subscribed to the Google
Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it,
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com
<
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
Post by Artur Szostak
;.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/068329c2-bd4c-468b-ac60-f7c138e18b37%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
g***@gmail.com
2018-11-27 14:49:10 UTC
Permalink
Tried out a bunch of them. Unfortunately, none of them are good for a
production scale.
Post by j***@godaddy.com
The beauty of Jenkins is that it's NOT built-in, but can be easily added
as a plugin (of which there are several).
Post by g***@gmail.com
I wish for a tool like Jenkins it had a backup and restore option built in.
Post by John Mellor
I have had restore issues with *ALL* of the backup plugins that I have
tried. The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink. This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.
Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).
Post by Artur Szostak
Just my opinion: but I would forget about the backup plugins in
Jenkins for full production backups. They all do a poor job when it
comes to a full backup for a production system that also needs high
availability. The best that can be done at the moment is to run
Jenkins on a file system or data store that is able to perform copy-
on-write snapshots. From within Jenkins you will want to setup a
simple job that triggers the copy-on-write snapshot when no other job
is running. Then use your normal production backup solutions to
backup the snapshot in the background.
Cheers
Artur
________________________________________
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore
We have a Jenkins instance running and we are trying to setup a
backup and restore mechanism in case of any mishaps. We are looking
for a full backup of the Jenkins and so decided to go with
PeriodicBackup plugin. With this plugin, the backup works
fine. But, we are trying to restore it in a completely new Jenkins
setup and not able to restore. When we pick the backup file that we
want to restore from, it says restoring, but nothing happens. Even
after waiting out for over 30 mins, nothing is restored. As a work
around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
(/var/lib/jenkins) folder to 'jenkins_backup' and created another
folder called 'jenkins' in the same location. In this folder, I
I didn't change the .owner file. I'm using the .owner file of the
new Jenkins only). Jenkins came up and I was able to see all the
configurations from the old one. Everything was fine. I let it like
that overnight and in the morning when I came and tried to access it,
Jenkins was way too slow and I was not able to save any new
configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI
to come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with
these are that they don't do full backup. But, with ThinBackup both
backup and restore works fine.
--
You received this message because you are subscribed to the Google
Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it,
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com
<
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
Post by Artur Szostak
;.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jeff Pearce
2018-11-27 17:24:54 UTC
Permalink
What issues are you seeing that prevent them from being good for production scale?

From: <jenkinsci-***@googlegroups.com> on behalf of "***@gmail.com" <***@gmail.com>
Reply-To: "jenkinsci-***@googlegroups.com" <jenkinsci-***@googlegroups.com>
Date: Tuesday, November 27, 2018 at 6:49 AM
To: Jenkins Users <jenkinsci-***@googlegroups.com>
Subject: Re: Jenkins Backup and Restore

Tried out a bunch of them. Unfortunately, none of them are good for a production scale.

On Monday, November 26, 2018 at 5:18:04 PM UTC-5, ***@godaddy.com wrote:
The beauty of Jenkins is that it's NOT built-in, but can be easily added as a plugin (of which there are several).

On Monday, November 26, 2018 at 1:41:19 PM UTC-8, ***@gmail.com wrote:
I wish for a tool like Jenkins it had a backup and restore option built in.

On Monday, November 26, 2018 at 4:35:04 PM UTC-5, john.mellor wrote:
I have had restore issues with *ALL* of the backup plugins that I have
tried. The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink. This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.

Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).
Post by Artur Szostak
Just my opinion: but I would forget about the backup plugins in
Jenkins for full production backups. They all do a poor job when it
comes to a full backup for a production system that also needs high
availability. The best that can be done at the moment is to run
Jenkins on a file system or data store that is able to perform copy-
on-write snapshots. From within Jenkins you will want to setup a
simple job that triggers the copy-on-write snapshot when no other job
is running. Then use your normal production backup solutions to
backup the snapshot in the background.
Cheers
Artur
________________________________________
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore
We have a Jenkins instance running and we are trying to setup a
backup and restore mechanism in case of any mishaps. We are looking
for a full backup of the Jenkins and so decided to go with
PeriodicBackup plugin. With this plugin, the backup works
fine. But, we are trying to restore it in a completely new Jenkins
setup and not able to restore. When we pick the backup file that we
want to restore from, it says restoring, but nothing happens. Even
after waiting out for over 30 mins, nothing is restored. As a work
around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
(/var/lib/jenkins) folder to 'jenkins_backup' and created another
folder called 'jenkins' in the same location. In this folder, I
I didn't change the .owner file. I'm using the .owner file of the
new Jenkins only). Jenkins came up and I was able to see all the
configurations from the old one. Everything was fine. I let it like
that overnight and in the morning when I came and tried to access it,
Jenkins was way too slow and I was not able to save any new
configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI
to come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with
these are that they don't do full backup. But, with ThinBackup both
backup and restore works fine.
--
You received this message because you are subscribed to the Google
Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it,
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
;.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com<mailto:jenkinsci-users+***@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/588AA5F3-DCE6-4516-B0C7-7E2C161E3D5C%40godaddy.com.
For more options, visit https://groups.google.com/d/optout.
g***@gmail.com
2018-11-27 18:28:32 UTC
Permalink
We are looking for a full backup of the JENKINS_HOME folder (automated).
We looked at SMC Sync plugin, ThinBackup, Backup and PeriodicBackup.
PeriodicBackup was the only one which offered both. But, with this we are
facing the following issues

1) The values that we set for "# of backups" and the "# of days to store
backup" does not work. It keeps creating more and more without deleting
the previous ones.
2) The restore feature does not work. I took one backup and tried to
restore a new Jenkins instance. It says "restoring backup" in the UI, but
no matter how long we wait, it doesn't restore. I had to manually extract
the file to the JENKINS_HOME folder.
Post by Jeff Pearce
What issues are you seeing that prevent them from being good for production scale?
*Date: *Tuesday, November 27, 2018 at 6:49 AM
*Subject: *Re: Jenkins Backup and Restore
Tried out a bunch of them. Unfortunately, none of them are good for a production scale.
The beauty of Jenkins is that it's NOT built-in, but can be easily added
as a plugin (of which there are several).
I wish for a tool like Jenkins it had a backup and restore option built in.
I have had restore issues with *ALL* of the backup plugins that I have
tried. The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink. This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.
Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).
Post by Artur Szostak
Just my opinion: but I would forget about the backup plugins in
Jenkins for full production backups. They all do a poor job when it
comes to a full backup for a production system that also needs high
availability. The best that can be done at the moment is to run
Jenkins on a file system or data store that is able to perform copy-
on-write snapshots. From within Jenkins you will want to setup a
simple job that triggers the copy-on-write snapshot when no other job
is running. Then use your normal production backup solutions to
backup the snapshot in the background.
Cheers
Artur
________________________________________
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore
We have a Jenkins instance running and we are trying to setup a
backup and restore mechanism in case of any mishaps. We are looking
for a full backup of the Jenkins and so decided to go with
PeriodicBackup plugin. With this plugin, the backup works
fine. But, we are trying to restore it in a completely new Jenkins
setup and not able to restore. When we pick the backup file that we
want to restore from, it says restoring, but nothing happens. Even
after waiting out for over 30 mins, nothing is restored. As a work
around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
(/var/lib/jenkins) folder to 'jenkins_backup' and created another
folder called 'jenkins' in the same location. In this folder, I
I didn't change the .owner file. I'm using the .owner file of the
new Jenkins only). Jenkins came up and I was able to see all the
configurations from the old one. Everything was fine. I let it like
that overnight and in the morning when I came and tried to access it,
Jenkins was way too slow and I was not able to save any new
configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI
to come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with
these are that they don't do full backup. But, with ThinBackup both
backup and restore works fine.
--
You received this message because you are subscribed to the Google
Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it,
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com
<
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
Post by Artur Szostak
;.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com
<https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/459109a3-0d6e-4780-8381-23c60a991516%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Artur Szostak
2018-11-28 19:08:18 UTC
Permalink
None of the plugins performs a copy-on-write style full backup of $JENKINS_HOME on the master (I am assuming workspaces are not included or stored somewhere else). Why copy-on-write: because I want close to zero down time in my Jenkins master when performing the backup. I also see no evidence of any of the backup plugins applying a proper quiesce strategy.


________________________________________
From: jenkinsci-***@googlegroups.com <jenkinsci-***@googlegroups.com> on behalf of Jeff Pearce <***@godaddy.com>
Sent: 27 November 2018 18:24
To: jenkinsci-***@googlegroups.com
Subject: Re: Jenkins Backup and Restore

What issues are you seeing that prevent them from being good for production scale?

From: <jenkinsci-***@googlegroups.com> on behalf of "***@gmail.com" <***@gmail.com>
Reply-To: "jenkinsci-***@googlegroups.com" <jenkinsci-***@googlegroups.com>
Date: Tuesday, November 27, 2018 at 6:49 AM
To: Jenkins Users <jenkinsci-***@googlegroups.com>
Subject: Re: Jenkins Backup and Restore

Tried out a bunch of them. Unfortunately, none of them are good for a production scale.

On Monday, November 26, 2018 at 5:18:04 PM UTC-5, ***@godaddy.com wrote:
The beauty of Jenkins is that it's NOT built-in, but can be easily added as a plugin (of which there are several).

On Monday, November 26, 2018 at 1:41:19 PM UTC-8, ***@gmail.com wrote:
I wish for a tool like Jenkins it had a backup and restore option built in.

On Monday, November 26, 2018 at 4:35:04 PM UTC-5, john.mellor wrote:
I have had restore issues with *ALL* of the backup plugins that I have
tried. The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink. This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.

Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).
Post by Artur Szostak
Just my opinion: but I would forget about the backup plugins in
Jenkins for full production backups. They all do a poor job when it
comes to a full backup for a production system that also needs high
availability. The best that can be done at the moment is to run
Jenkins on a file system or data store that is able to perform copy-
on-write snapshots. From within Jenkins you will want to setup a
simple job that triggers the copy-on-write snapshot when no other job
is running. Then use your normal production backup solutions to
backup the snapshot in the background.
Cheers
Artur
________________________________________
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore
We have a Jenkins instance running and we are trying to setup a
backup and restore mechanism in case of any mishaps. We are looking
for a full backup of the Jenkins and so decided to go with
PeriodicBackup plugin. With this plugin, the backup works
fine. But, we are trying to restore it in a completely new Jenkins
setup and not able to restore. When we pick the backup file that we
want to restore from, it says restoring, but nothing happens. Even
after waiting out for over 30 mins, nothing is restored. As a work
around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
(/var/lib/jenkins) folder to 'jenkins_backup' and created another
folder called 'jenkins' in the same location. In this folder, I
I didn't change the .owner file. I'm using the .owner file of the
new Jenkins only). Jenkins came up and I was able to see all the
configurations from the old one. Everything was fine. I let it like
that overnight and in the morning when I came and tried to access it,
Jenkins was way too slow and I was not able to save any new
configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI
to come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with
these are that they don't do full backup. But, with ThinBackup both
backup and restore works fine.
--
You received this message because you are subscribed to the Google
Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it,
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
;.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com<mailto:jenkinsci-users+***@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/f5625308-1145-4d77-b021-62be8a779592%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com<mailto:jenkinsci-users+***@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/588AA5F3-DCE6-4516-B0C7-7E2C161E3D5C%40godaddy.com<https://groups.google.com/d/msgid/jenkinsci-users/588AA5F3-DCE6-4516-B0C7-7E2C161E3D5C%40godaddy.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b390f03cc61741098a8da940c4badeaf%40partner.eso.org.
For more options, visit https://groups.google.com/d/optout.
j***@godaddy.com
2018-11-26 21:43:27 UTC
Permalink
Have you considered sending a PR to (or at least filing a Github issue for)
the plugins in question?



It's probably an easy fix to handle symlinks



Best

Jeff
Post by John Mellor
I have had restore issues with *ALL* of the backup plugins that I have
tried. The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink. This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.
Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).
Post by Artur Szostak
Just my opinion: but I would forget about the backup plugins in
Jenkins for full production backups. They all do a poor job when it
comes to a full backup for a production system that also needs high
availability. The best that can be done at the moment is to run
Jenkins on a file system or data store that is able to perform copy-
on-write snapshots. From within Jenkins you will want to setup a
simple job that triggers the copy-on-write snapshot when no other job
is running. Then use your normal production backup solutions to
backup the snapshot in the background.
Cheers
Artur
________________________________________
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore
We have a Jenkins instance running and we are trying to setup a
backup and restore mechanism in case of any mishaps. We are looking
for a full backup of the Jenkins and so decided to go with
PeriodicBackup plugin. With this plugin, the backup works
fine. But, we are trying to restore it in a completely new Jenkins
setup and not able to restore. When we pick the backup file that we
want to restore from, it says restoring, but nothing happens. Even
after waiting out for over 30 mins, nothing is restored. As a work
around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
(/var/lib/jenkins) folder to 'jenkins_backup' and created another
folder called 'jenkins' in the same location. In this folder, I
I didn't change the .owner file. I'm using the .owner file of the
new Jenkins only). Jenkins came up and I was able to see all the
configurations from the old one. Everything was fine. I let it like
that overnight and in the morning when I came and tried to access it,
Jenkins was way too slow and I was not able to save any new
configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI
to come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with
these are that they don't do full backup. But, with ThinBackup both
backup and restore works fine.
--
You received this message because you are subscribed to the Google
Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it,
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com
<
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
Post by Artur Szostak
;.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e16c5852-8d61-4dab-b717-81e3b6e7849e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
g***@gmail.com
2018-11-27 14:48:31 UTC
Permalink
The plugin team's been unresponsive for ages. No help there.
Post by j***@godaddy.com
Have you considered sending a PR to (or at least filing a Github issue
for) the plugins in question?
It's probably an easy fix to handle symlinks
Best
Jeff
Post by John Mellor
I have had restore issues with *ALL* of the backup plugins that I have
tried. The problem is that they are using stupid mechanisms to copy
the filesystem that cannot correctly copy a symlink. This breaks the
LastSuccessfulBuild and LastUnsuccessfulBuild and LastFailedBuild
symlinks, causing lots of secondary issues.
Doing my own using cpio or rsync in a cronjob only messes up copying
very large files that exceed the archive limits (4GB for cpio for
instance).
Post by Artur Szostak
Just my opinion: but I would forget about the backup plugins in
Jenkins for full production backups. They all do a poor job when it
comes to a full backup for a production system that also needs high
availability. The best that can be done at the moment is to run
Jenkins on a file system or data store that is able to perform copy-
on-write snapshots. From within Jenkins you will want to setup a
simple job that triggers the copy-on-write snapshot when no other job
is running. Then use your normal production backup solutions to
backup the snapshot in the background.
Cheers
Artur
________________________________________
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore
We have a Jenkins instance running and we are trying to setup a
backup and restore mechanism in case of any mishaps. We are looking
for a full backup of the Jenkins and so decided to go with
PeriodicBackup plugin. With this plugin, the backup works
fine. But, we are trying to restore it in a completely new Jenkins
setup and not able to restore. When we pick the backup file that we
want to restore from, it says restoring, but nothing happens. Even
after waiting out for over 30 mins, nothing is restored. As a work
around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
(/var/lib/jenkins) folder to 'jenkins_backup' and created another
folder called 'jenkins' in the same location. In this folder, I
I didn't change the .owner file. I'm using the .owner file of the
new Jenkins only). Jenkins came up and I was able to see all the
configurations from the old one. Everything was fine. I let it like
that overnight and in the morning when I came and tried to access it,
Jenkins was way too slow and I was not able to save any new
configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI
to come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with
these are that they don't do full backup. But, with ThinBackup both
backup and restore works fine.
--
You received this message because you are subscribed to the Google
Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it,
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com
<
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
Post by Artur Szostak
;.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/f4851afe-87a4-4a8c-8499-9d65388e8cad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
g***@gmail.com
2018-11-26 21:42:22 UTC
Permalink
Our cloud is Azure. Do you have any suggestions for this?
Post by Artur Szostak
Just my opinion: but I would forget about the backup plugins in Jenkins
for full production backups. They all do a poor job when it comes to a full
backup for a production system that also needs high availability. The best
that can be done at the moment is to run Jenkins on a file system or data
store that is able to perform copy-on-write snapshots. From within Jenkins
you will want to setup a simple job that triggers the copy-on-write
snapshot when no other job is running. Then use your normal production
backup solutions to backup the snapshot in the background.
Cheers
Artur
________________________________________
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore
We have a Jenkins instance running and we are trying to setup a backup and
restore mechanism in case of any mishaps. We are looking for a full backup
of the Jenkins and so decided to go with PeriodicBackup plugin. With this
plugin, the backup works fine. But, we are trying to restore it in a
completely new Jenkins setup and not able to restore. When we pick the
backup file that we want to restore from, it says restoring, but nothing
happens. Even after waiting out for over 30 mins, nothing is restored. As
a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
(/var/lib/jenkins) folder to 'jenkins_backup' and created another folder
called 'jenkins' in the same location. In this folder, I copied all the
files from the backup and then started Jenkins (NOTE: I didn't change the
.owner file. I'm using the .owner file of the new Jenkins only). Jenkins
came up and I was able to see all the configurations from the old one.
Everything was fine. I let it like that overnight and in the morning when
I came and tried to access it, Jenkins was way too slow and I was not able
to save any new configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI to
come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with these are
that they don't do full backup. But, with ThinBackup both backup and
restore works fine.
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com
<
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/7c46683a-fbfb-4d79-b470-10246e8999d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Artur Szostak
2018-11-28 18:59:39 UTC
Permalink
Do you mean your Jenkins master is hosted on a VM in Azure or the build nodes?
I would not backup the build nodes. I would treat them as volatile and only focus on backing up the master. Unless you have a good reason for backing up the build nodes also.

As for the Jenkins master, I do not know much about Azure, but does it have any kind of copy-on-write snapshot mechanism that you could trigger from within the VM? If yes, then you could call that mechanism from within a periodic backup job within Jenkins, but make sure the job only executes when nothing else is running on Jenkins.

In our case we use Btrfs for the filesystem, so we call something like the following from within a Jenkins job:

sudo btrfs subvolume snapshot <jenkins_home> <snapshot_path>

To make sure the job only runs when nothing else is running we take advantage of the following plugin:

https://plugins.jenkins.io/build-blocker-plugin

and then just use '.*' for the regular expression pattern.
After the snapshot is complete, the snapshot itself is backed up to tape with a commercial solution. I believe our IT department does this on the block level.

Cheers

Artur

________________________________________
From: jenkinsci-***@googlegroups.com <jenkinsci-***@googlegroups.com> on behalf of ***@gmail.com <***@gmail.com>
Sent: 26 November 2018 22:42
To: Jenkins Users
Subject: Re: Jenkins Backup and Restore

Our cloud is Azure. Do you have any suggestions for this?

On Monday, November 26, 2018 at 4:13:06 PM UTC-5, Artur Szostak wrote:
Just my opinion: but I would forget about the backup plugins in Jenkins for full production backups. They all do a poor job when it comes to a full backup for a production system that also needs high availability. The best that can be done at the moment is to run Jenkins on a file system or data store that is able to perform copy-on-write snapshots. From within Jenkins you will want to setup a simple job that triggers the copy-on-write snapshot when no other job is running. Then use your normal production backup solutions to backup the snapshot in the background.

Cheers

Artur

________________________________________
From: ***@googlegroups.com <***@googlegroups.com> on behalf of ***@gmail.com <***@gmail.com>
Sent: 01 November 2018 21:36
To: Jenkins Users
Subject: Jenkins Backup and Restore

We have a Jenkins instance running and we are trying to setup a backup and restore mechanism in case of any mishaps. We are looking for a full backup of the Jenkins and so decided to go with PeriodicBackup plugin. With this plugin, the backup works fine. But, we are trying to restore it in a completely new Jenkins setup and not able to restore. When we pick the backup file that we want to restore from, it says restoring, but nothing happens. Even after waiting out for over 30 mins, nothing is restored. As a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME' (/var/lib/jenkins) folder to 'jenkins_backup' and created another folder called 'jenkins' in the same location. In this folder, I copied all the files from the backup and then started Jenkins (NOTE: I didn't change the .owner file. I'm using the .owner file of the new Jenkins only). Jenkins came up and I was able to see all the configurations from the old one. Everything was fine. I let it like that overnight and in the morning when I came and tried to access it, Jenkins was way too slow and I was not able to save any new configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI to come up. Am I doing anything wrong here? Any suggestions for my requirement?

P.S. - I tried SCMSync plugin and ThinBackup. But, problem with these are that they don't do full backup. But, with ThinBackup both backup and restore works fine.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-***@googlegroups.com<mailto:jenkinsci-users+***@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com<mailto:jenkinsci-users+***@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/7c46683a-fbfb-4d79-b470-10246e8999d6%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/7c46683a-fbfb-4d79-b470-10246e8999d6%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/c643a72943554a04bff8a5d0064f18d7%40partner.eso.org.
For more options, visit https://groups.google.com/d/optout.
RAJENDRA PRASAD
2018-11-27 01:03:08 UTC
Permalink
Hi ,
I have another solution, just document the list of plugins in old Jenkins
server and instead of backup and restore plugins just reinstall plugins on
new Jenkins server.

Thanks,
Rajendra
Post by g***@gmail.com
We have a Jenkins instance running and we are trying to setup a backup and
restore mechanism in case of any mishaps. We are looking for a full backup
of the Jenkins and so decided to go with PeriodicBackup plugin. With this
plugin, the backup works fine. But, we are trying to restore it in a
completely new Jenkins setup and not able to restore. When we pick the
backup file that we want to restore from, it says restoring, but nothing
happens. Even after waiting out for over 30 mins, nothing is restored. As
a work around, I stopped the Jenkins service, renamed the 'JENKINS_HOME'
(/var/lib/jenkins) folder to 'jenkins_backup' and created another folder
called 'jenkins' in the same location. In this folder, I copied all the
files from the backup and then started Jenkins (NOTE: I didn't change the
.owner file. I'm using the .owner file of the new Jenkins only). Jenkins
came up and I was able to see all the configurations from the old one.
Everything was fine. I let it like that overnight and in the morning when
I came and tried to access it, Jenkins was way too slow and I was not able
to save any new configuration. When I try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI to
come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with these are
that they don't do full backup. But, with ThinBackup both backup and
restore works fine.
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com
<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAMrg02T5QPm9WRN2dpOOEtTZUnjtiqXAEONDVt1ahn4nh-ZM%2Bg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
g***@gmail.com
2018-11-27 14:50:41 UTC
Permalink
I'm not sure what you are trying to say?

On Monday, November 26, 2018 at 8:03:34 PM UTC-5, rajendraprasad reddy
Post by RAJENDRA PRASAD
Hi ,
I have another solution, just document the list of plugins in old Jenkins
server and instead of backup and restore plugins just reinstall plugins on
new Jenkins server.
Thanks,
Rajendra
Post by g***@gmail.com
We have a Jenkins instance running and we are trying to setup a backup
and restore mechanism in case of any mishaps. We are looking for a full
backup of the Jenkins and so decided to go with PeriodicBackup plugin.
With this plugin, the backup works fine. But, we are trying to restore it
in a completely new Jenkins setup and not able to restore. When we pick
the backup file that we want to restore from, it says restoring, but
nothing happens. Even after waiting out for over 30 mins, nothing is
restored. As a work around, I stopped the Jenkins service, renamed the
'JENKINS_HOME' (/var/lib/jenkins) folder to 'jenkins_backup' and created
another folder called 'jenkins' in the same location. In this folder, I
copied all the files from the backup and then started Jenkins (NOTE: I
didn't change the .owner file. I'm using the .owner file of the new
Jenkins only). Jenkins came up and I was able to see all the
configurations from the old one. Everything was fine. I let it like that
overnight and in the morning when I came and tried to access it, Jenkins
was way too slow and I was not able to save any new configuration. When I
try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI to
come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with these
are that they don't do full backup. But, with ThinBackup both backup and
restore works fine.
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com
<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/9d3fad8e-7bbe-4472-86dd-57ef3d0da960%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Mark Waite
2018-11-27 17:39:46 UTC
Permalink
He's saying that if you don't need to backup and restore your job
definitions (possibly they are all multibranch Pipeline jobs that are easy
to recreate, or they are all defined with the Job DSL plugin so they will
be automatically recreated or they are defined in a Docker image) and don't
need to backup and restore your credential definitions (same reasons) and
don't need to backup and restore other configuration settings (possibly
because you're using Configuration as Code plugin to store your
configuration in a source control system), then there are backup concepts
that use a source control system as your backup system.

In your use case, I suspect you want to use a backup.

Mark Waite
Post by g***@gmail.com
I'm not sure what you are trying to say?
On Monday, November 26, 2018 at 8:03:34 PM UTC-5, rajendraprasad reddy
Post by RAJENDRA PRASAD
Hi ,
I have another solution, just document the list of plugins in old Jenkins
server and instead of backup and restore plugins just reinstall plugins on
new Jenkins server.
Thanks,
Rajendra
Post by g***@gmail.com
We have a Jenkins instance running and we are trying to setup a backup
and restore mechanism in case of any mishaps. We are looking for a full
backup of the Jenkins and so decided to go with PeriodicBackup plugin.
With this plugin, the backup works fine. But, we are trying to restore it
in a completely new Jenkins setup and not able to restore. When we pick
the backup file that we want to restore from, it says restoring, but
nothing happens. Even after waiting out for over 30 mins, nothing is
restored. As a work around, I stopped the Jenkins service, renamed the
'JENKINS_HOME' (/var/lib/jenkins) folder to 'jenkins_backup' and created
another folder called 'jenkins' in the same location. In this folder, I
copied all the files from the backup and then started Jenkins (NOTE: I
didn't change the .owner file. I'm using the .owner file of the new
Jenkins only). Jenkins came up and I was able to see all the
configurations from the old one. Everything was fine. I let it like that
overnight and in the morning when I came and tried to access it, Jenkins
was way too slow and I was not able to save any new configuration. When I
try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI to
come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with these
are that they don't do full backup. But, with ThinBackup both backup and
restore works fine.
--
You received this message because you are subscribed to the Google
Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com
<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/9d3fad8e-7bbe-4472-86dd-57ef3d0da960%40googlegroups.com
<https://groups.google.com/d/msgid/jenkinsci-users/9d3fad8e-7bbe-4472-86dd-57ef3d0da960%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Thanks!
Mark Waite
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtH-ydL8JkmnFurJ06s%2BEn-b%3DWaNmb_KT%3DBFiz6mbZK8QQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
g***@gmail.com
2018-11-27 18:30:53 UTC
Permalink
I'm relatively new to all these and I'm just now realizing there are this
many ways to use Jenkins (sorry for being dumb). YES!!! like you said, we
want to use a backup mechanism as we don't have any of these in place.
Where can I find documentation or study materials for the stuff you
mentioned here? Would really appreciate if you can point me to some.
Post by Mark Waite
He's saying that if you don't need to backup and restore your job
definitions (possibly they are all multibranch Pipeline jobs that are easy
to recreate, or they are all defined with the Job DSL plugin so they will
be automatically recreated or they are defined in a Docker image) and don't
need to backup and restore your credential definitions (same reasons) and
don't need to backup and restore other configuration settings (possibly
because you're using Configuration as Code plugin to store your
configuration in a source control system), then there are backup concepts
that use a source control system as your backup system.
In your use case, I suspect you want to use a backup.
Mark Waite
Post by g***@gmail.com
I'm not sure what you are trying to say?
On Monday, November 26, 2018 at 8:03:34 PM UTC-5, rajendraprasad reddy
Post by RAJENDRA PRASAD
Hi ,
I have another solution, just document the list of plugins in old
Jenkins server and instead of backup and restore plugins just reinstall
plugins on new Jenkins server.
Thanks,
Rajendra
Post by g***@gmail.com
We have a Jenkins instance running and we are trying to setup a backup
and restore mechanism in case of any mishaps. We are looking for a full
backup of the Jenkins and so decided to go with PeriodicBackup plugin.
With this plugin, the backup works fine. But, we are trying to restore it
in a completely new Jenkins setup and not able to restore. When we pick
the backup file that we want to restore from, it says restoring, but
nothing happens. Even after waiting out for over 30 mins, nothing is
restored. As a work around, I stopped the Jenkins service, renamed the
'JENKINS_HOME' (/var/lib/jenkins) folder to 'jenkins_backup' and created
another folder called 'jenkins' in the same location. In this folder, I
copied all the files from the backup and then started Jenkins (NOTE: I
didn't change the .owner file. I'm using the .owner file of the new
Jenkins only). Jenkins came up and I was able to see all the
configurations from the old one. Everything was fine. I let it like that
overnight and in the morning when I came and tried to access it, Jenkins
was way too slow and I was not able to save any new configuration. When I
try I'm getting
java.io.ioexception: too many open files jenkins
error. I restarted the jenkins and it's taking too long for the UI to
come up. Am I doing anything wrong here? Any suggestions for my
requirement?
P.S. - I tried SCMSync plugin and ThinBackup. But, problem with these
are that they don't do full backup. But, with ThinBackup both backup and
restore works fine.
--
You received this message because you are subscribed to the Google
Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com
<https://groups.google.com/d/msgid/jenkinsci-users/0c58b4fa-4e8b-4c53-9ca9-b99397078fae%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/9d3fad8e-7bbe-4472-86dd-57ef3d0da960%40googlegroups.com
<https://groups.google.com/d/msgid/jenkinsci-users/9d3fad8e-7bbe-4472-86dd-57ef3d0da960%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Thanks!
Mark Waite
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/ce0af9d3-07d0-4de3-915f-ecff4bf06e47%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Loading...