Pipeline checkout persistence issue

[James Chapman]

Hello Jenkins users,

I have a pipeline job that run across a whole lot of different platforms.
The job checks out source code on each node, builds on each node, runs
tests on each node and then completes. Pretty straightforward!
For some reason, when the checkout finishes, all the files that were just
checked out get deleted and the build then fails because the required
buildproj.pl file is missing. All that remains after the checkout step is
the Jenkinsfiles.

Can anyone help me understand why the checked out files are being deleted?

Thanks
James



In the *project root* I have:

build/
jenkins/Jenkinsfile_svn_checkout
source/
buildproj.pl
Jenkinsfile


*Jenkinsfile_svn_checkout:*

checkout poll: false,
scm: [$class: 'SubversionSCM',
additionalCredentials: [],
excludedCommitMessages: '',
excludedRegions: '',
excludedRevprop: '',
excludedUsers: 'userXYZ',
filterChangelog: true,
ignoreDirPropChanges: true,
includedRegions: '',
locations: [[cancelProcessOnExternalsFail: false,
credentialsId: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',
depthOption: 'infinity',
ignoreExternalsOption: false,
local: '.',
remote: "${g_subversion_url}/${params.SUBVERSION_BRANCH}"]],
quietOperation: false,
workspaceUpdater: [$class: 'UpdateUpdater']
]


*Jenkinsfile*:

pipeline {

agent none

parameters {
booleanParam(name: 'BUILD_WINDOWS_32', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_WINDOWS_64', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_LINUX_32', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_LINUX_64', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_SOLARIS_10', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_FREEBSD_10', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_MAC_OSX_64', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_AIX_PPC', defaultValue: true,
description: '')
string(name: 'SUBVERSION_BRANCH', defaultValue: 'trunk',
description: '')
}

environment {

String g_subversion_url = 'https://svn.project.url'
int m_win64_code = 0
int m_win32_code = 0

}

options {
buildDiscarder(logRotator(numToKeepStr: '10'))
timestamps()
}

triggers {
pollSCM('H/5 * * * *')
}

stages {

stage('Checkout Build Test') {
failFast false
parallel {
stage('Windows 64') {
agent {
node {
label 'WIN64'
customWorkspace 'D:\\JENKINS\\Project_pipeline'
}
}
steps {
script {
if (params.BUILD_WINDOWS_64) {
load 'jenkins/Jenkinsfile_svn_checkout'
m_win64_code = bat (returnStatus: true,
script: '''perl buildproj.pl win64''')
}
}
}
}
stage('Windows 32') {
agent {
node {
label 'WIN32'
customWorkspace 'D:\\JENKINS\\Project_pipeline'
}
}
steps {
script {
if (params.BUILD_WINDOWS_32) {
load 'jenkins/Jenkinsfile_svn_checkout'
m_win32_code = bat (returnStatus: true,
script: '''perl buildproj.pl win32''')
}
}
}
}
// ... more stages for other platforms
}
}

} // stages

post {

always {
echo "Unit tests completed"
}

success {
echo "All test passed"
}

failure {
echo 'Unit tests failed'
}
}
} // pipeline

[Andrew Bayer]

You might want to add "skipDefaultCheckout(true)" to your options - it
could be that the automatic checkout that happens whenever you enter an
agent is messing things up.

A.

Post by James Chapman
Hello Jenkins users,
I have a pipeline job that run across a whole lot of different platforms.
The job checks out source code on each node, builds on each node, runs
tests on each node and then completes. Pretty straightforward!
For some reason, when the checkout finishes, all the files that were just
checked out get deleted and the build then fails because the required
buildproj.pl file is missing. All that remains after the checkout step is
the Jenkinsfiles.
Can anyone help me understand why the checked out files are being deleted?
Thanks
James
build/
jenkins/Jenkinsfile_svn_checkout
source/
buildproj.pl
Jenkinsfile
*Jenkinsfile_svn_checkout:*
checkout poll: false,
scm: [$class: 'SubversionSCM',
additionalCredentials: [],
excludedCommitMessages: '',
excludedRegions: '',
excludedRevprop: '',
excludedUsers: 'userXYZ',
filterChangelog: true,
ignoreDirPropChanges: true,
includedRegions: '',
locations: [[cancelProcessOnExternalsFail: false,
credentialsId: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',
depthOption: 'infinity',
ignoreExternalsOption: false,
local: '.',
remote: "${g_subversion_url}/${params.SUBVERSION_BRANCH}"]],
quietOperation: false,
workspaceUpdater: [$class: 'UpdateUpdater']
]
pipeline {
agent none
parameters {
booleanParam(name: 'BUILD_WINDOWS_32', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_WINDOWS_64', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_LINUX_32', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_LINUX_64', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_SOLARIS_10', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_FREEBSD_10', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_MAC_OSX_64', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_AIX_PPC', defaultValue: true,
description: '')
string(name: 'SUBVERSION_BRANCH', defaultValue: 'trunk',
description: '')
}
environment {
String g_subversion_url = 'https://svn.project.url'
int m_win64_code = 0
int m_win32_code = 0
}
options {
buildDiscarder(logRotator(numToKeepStr: '10'))
timestamps()
}
triggers {
pollSCM('H/5 * * * *')
}
stages {
stage('Checkout Build Test') {
failFast false
parallel {
stage('Windows 64') {
agent {
node {
label 'WIN64'
customWorkspace 'D:\\JENKINS\\Project_pipeline'
}
}
steps {
script {
if (params.BUILD_WINDOWS_64) {
load 'jenkins/Jenkinsfile_svn_checkout'
m_win64_code = bat (returnStatus: true,
script: '''perl buildproj.pl win64''')
}
}
}
}
stage('Windows 32') {
agent {
node {
label 'WIN32'
customWorkspace 'D:\\JENKINS\\Project_pipeline'
}
}
steps {
script {
if (params.BUILD_WINDOWS_32) {
load 'jenkins/Jenkinsfile_svn_checkout'
m_win32_code = bat (returnStatus: true,
script: '''perl buildproj.pl win32''')
}
}
}
}
// ... more stages for other platforms
}
}
} // stages
post {
always {
echo "Unit tests completed"
}
success {
echo "All test passed"
}
failure {
echo 'Unit tests failed'
}
}
} // pipeline
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/CAHvkzy%3DEtB_1umVBgUUC5rL8g-x%2BC9Zu1M%3DpB7St1mBazoWqbQ%40mail.gmail.com
<https://groups.google.com/d/msgid/jenkinsci-users/CAHvkzy%3DEtB_1umVBgUUC5rL8g-x%2BC9Zu1M%3DpB7St1mBazoWqbQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[James Chapman]

Yeah, the default checkout is definitely the problem, even though it's
"lightweight", it isn't. When I then perform a chekout step, once complete
the default checkout reverts the directory and it then becomes
"lightweight", so only the Jenkinsfiles remain and the source I'm trying to
build evaporates.

I added the skipDefaultCheckout() directive (thanks for that, Andrew, I
didn't know it existed) but now the manual checkout step fails because it
is in a seperate file. If I move the checkout into the primary Jenkinsfile
it works, I can live with that for now, albeit a little bit annoying
because the checkout is common.

The other option that works is to completely remove the call to checkout
because it does a full checkout by default.

Either way, the default checkout seems to be the cause of my headaches and
I now know how to get around it, thanks.

James

Post by Andrew Bayer
You might want to add "skipDefaultCheckout(true)" to your options - it
could be that the automatic checkout that happens whenever you enter an
agent is messing things up.
A.

Post by James Chapman
Hello Jenkins users,
I have a pipeline job that run across a whole lot of different platforms.
The job checks out source code on each node, builds on each node, runs
tests on each node and then completes. Pretty straightforward!
For some reason, when the checkout finishes, all the files that were just
checked out get deleted and the build then fails because the required
buildproj.pl file is missing. All that remains after the checkout step
is the Jenkinsfiles.
Can anyone help me understand why the checked out files are being deleted?
Thanks
James
build/
jenkins/Jenkinsfile_svn_checkout
source/
buildproj.pl
Jenkinsfile
*Jenkinsfile_svn_checkout:*
checkout poll: false,
scm: [$class: 'SubversionSCM',
additionalCredentials: [],
excludedCommitMessages: '',
excludedRegions: '',
excludedRevprop: '',
excludedUsers: 'userXYZ',
filterChangelog: true,
ignoreDirPropChanges: true,
includedRegions: '',
locations: [[cancelProcessOnExternalsFail: false,
credentialsId: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',
depthOption: 'infinity',
ignoreExternalsOption: false,
local: '.',
remote: "${g_subversion_url}/${params.SUBVERSION_BRANCH}"]],
quietOperation: false,
workspaceUpdater: [$class: 'UpdateUpdater']
]
pipeline {
agent none
parameters {
booleanParam(name: 'BUILD_WINDOWS_32', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_WINDOWS_64', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_LINUX_32', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_LINUX_64', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_SOLARIS_10', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_FREEBSD_10', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_MAC_OSX_64', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_AIX_PPC', defaultValue: true,
description: '')
string(name: 'SUBVERSION_BRANCH', defaultValue: 'trunk',
description: '')
}
environment {
String g_subversion_url = 'https://svn.project.url'
int m_win64_code = 0
int m_win32_code = 0
}
options {
buildDiscarder(logRotator(numToKeepStr: '10'))
timestamps()
}
triggers {
pollSCM('H/5 * * * *')
}
stages {
stage('Checkout Build Test') {
failFast false
parallel {
stage('Windows 64') {
agent {
node {
label 'WIN64'
customWorkspace
'D:\\JENKINS\\Project_pipeline'
}
}
steps {
script {
if (params.BUILD_WINDOWS_64) {
load 'jenkins/Jenkinsfile_svn_checkout'
m_win64_code = bat (returnStatus: true,
script: '''perl buildproj.pl win64''')
}
}
}
}
stage('Windows 32') {
agent {
node {
label 'WIN32'
customWorkspace
'D:\\JENKINS\\Project_pipeline'
}
}
steps {
script {
if (params.BUILD_WINDOWS_32) {
load 'jenkins/Jenkinsfile_svn_checkout'
m_win32_code = bat (returnStatus: true,
script: '''perl buildproj.pl win32''')
}
}
}
}
// ... more stages for other platforms
}
}
} // stages
post {
always {
echo "Unit tests completed"
}
success {
echo "All test passed"
}
failure {
echo 'Unit tests failed'
}
}
} // pipeline
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/CAHvkzy%3DEtB_1umVBgUUC5rL8g-x%2BC9Zu1M%3DpB7St1mBazoWqbQ%40mail.gmail.com
<https://groups.google.com/d/msgid/jenkinsci-users/CAHvkzy%3DEtB_1umVBgUUC5rL8g-x%2BC9Zu1M%3DpB7St1mBazoWqbQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/CAPbPdOaR-yxzKa2S3ncFuXz%3DVsvCnc8_18EEqU6nSFgZKMhzag%40mail.gmail.com
<https://groups.google.com/d/msgid/jenkinsci-users/CAPbPdOaR-yxzKa2S3ncFuXz%3DVsvCnc8_18EEqU6nSFgZKMhzag%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[Andrew Bayer]

I'd advise moving the checkout logic into a shared library, fwiw.

A.

Post by James Chapman
Yeah, the default checkout is definitely the problem, even though it's
"lightweight", it isn't. When I then perform a chekout step, once complete
the default checkout reverts the directory and it then becomes
"lightweight", so only the Jenkinsfiles remain and the source I'm trying to
build evaporates.
I added the skipDefaultCheckout() directive (thanks for that, Andrew, I
didn't know it existed) but now the manual checkout step fails because it
is in a seperate file. If I move the checkout into the primary Jenkinsfile
it works, I can live with that for now, albeit a little bit annoying
because the checkout is common.
The other option that works is to completely remove the call to checkout
because it does a full checkout by default.
Either way, the default checkout seems to be the cause of my headaches and
I now know how to get around it, thanks.
James

Post by Andrew Bayer
You might want to add "skipDefaultCheckout(true)" to your options - it
could be that the automatic checkout that happens whenever you enter an
agent is messing things up.
A.

Post by James Chapman
Hello Jenkins users,
I have a pipeline job that run across a whole lot of different
platforms. The job checks out source code on each node, builds on each
node, runs tests on each node and then completes. Pretty straightforward!
For some reason, when the checkout finishes, all the files that were
just checked out get deleted and the build then fails because the required
buildproj.pl file is missing. All that remains after the checkout step
is the Jenkinsfiles.
Can anyone help me understand why the checked out files are being deleted?
Thanks
James
build/
jenkins/Jenkinsfile_svn_checkout
source/
buildproj.pl
Jenkinsfile
*Jenkinsfile_svn_checkout:*
checkout poll: false,
scm: [$class: 'SubversionSCM',
additionalCredentials: [],
excludedCommitMessages: '',
excludedRegions: '',
excludedRevprop: '',
excludedUsers: 'userXYZ',
filterChangelog: true,
ignoreDirPropChanges: true,
includedRegions: '',
locations: [[cancelProcessOnExternalsFail: false,
credentialsId: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',
depthOption: 'infinity',
ignoreExternalsOption: false,
local: '.',
remote: "${g_subversion_url}/${params.SUBVERSION_BRANCH}"]],
quietOperation: false,
workspaceUpdater: [$class: 'UpdateUpdater']
]
pipeline {
agent none
parameters {
booleanParam(name: 'BUILD_WINDOWS_32', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_WINDOWS_64', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_LINUX_32', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_LINUX_64', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_SOLARIS_10', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_FREEBSD_10', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_MAC_OSX_64', defaultValue: true,
description: '')
booleanParam(name: 'BUILD_AIX_PPC', defaultValue: true,
description: '')
string(name: 'SUBVERSION_BRANCH', defaultValue: 'trunk',
description: '')
}
environment {
String g_subversion_url = 'https://svn.project.url'
int m_win64_code = 0
int m_win32_code = 0
}
options {
buildDiscarder(logRotator(numToKeepStr: '10'))
timestamps()
}
triggers {
pollSCM('H/5 * * * *')
}
stages {
stage('Checkout Build Test') {
failFast false
parallel {
stage('Windows 64') {
agent {
node {
label 'WIN64'
customWorkspace
'D:\\JENKINS\\Project_pipeline'
}
}
steps {
script {
if (params.BUILD_WINDOWS_64) {
load 'jenkins/Jenkinsfile_svn_checkout'
m_win64_code = bat (returnStatus: true,
script: '''perl buildproj.pl win64''')
}
}
}
}
stage('Windows 32') {
agent {
node {
label 'WIN32'
customWorkspace
'D:\\JENKINS\\Project_pipeline'
}
}
steps {
script {
if (params.BUILD_WINDOWS_32) {
load 'jenkins/Jenkinsfile_svn_checkout'
m_win32_code = bat (returnStatus: true,
script: '''perl buildproj.pl win32''')
}
}
}
}
// ... more stages for other platforms
}
}
} // stages
post {
always {
echo "Unit tests completed"
}
success {
echo "All test passed"
}
failure {
echo 'Unit tests failed'
}
}
} // pipeline
--
You received this message because you are subscribed to the Google
Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/CAHvkzy%3DEtB_1umVBgUUC5rL8g-x%2BC9Zu1M%3DpB7St1mBazoWqbQ%40mail.gmail.com
<https://groups.google.com/d/msgid/jenkinsci-users/CAHvkzy%3DEtB_1umVBgUUC5rL8g-x%2BC9Zu1M%3DpB7St1mBazoWqbQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/CAPbPdOaR-yxzKa2S3ncFuXz%3DVsvCnc8_18EEqU6nSFgZKMhzag%40mail.gmail.com
<https://groups.google.com/d/msgid/jenkinsci-users/CAPbPdOaR-yxzKa2S3ncFuXz%3DVsvCnc8_18EEqU6nSFgZKMhzag%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/CAHvkzymJN28%2BnaO%3DEvEscDRaa%2BFetSMbw0Y2C2gnbh9GhRxnXA%40mail.gmail.com
<https://groups.google.com/d/msgid/jenkinsci-users/CAHvkzymJN28%2BnaO%3DEvEscDRaa%2BFetSMbw0Y2C2gnbh9GhRxnXA%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.