Discussion:
Kubernetes Plugin - Fails When Executing Commands on Second Container
Carlos Sanchez
2018-12-10 16:10:04 UTC
Permalink
You may be hitting the limit of concurrent connections to k8s api, see
https://github.com/jenkinsci/kubernetes-plugin/blob/master/CHANGELOG.md#1136

On Mon, Dec 10, 2018 at 8:01 AM Simon Young <
Hi,
We are trying to use the Kubernetes Plugin to run tests in a different
container - but *almost* every time we try and execute a command in the
second container, the build fails immediately. I say "almost" because the
build always succeeds after restarting the Jenkins master (or possibly
after it's been idle for a long time).
def label = "k8s-test-${UUID.randomUUID().toString()}"
podTemplate(
cloud: 'kubernetes-test.k8s.local',
namespace: 'mynamespace',
label: label,
yaml: """
apiVersion: v1
kind: Pod
- name: busybox
image: busybox
command: ['cat']
tty: true
"""
) {
node (label) {
stage ('Get Agent Info') {
sh "java -version"
}
stage ('Run tests') {
container('busybox') {
sh "echo foo"
sh "netstat -tln"
sh "df -h"
}
}
stage ('Grab Logs') {
containerLog('jnlp')
}
}
}
0, active threads = 0, queued tasks = 0, completed tasks = 3]
* Both containers are successfully deployed to the same pod, as expected.
* If we don't try and run the shell commands on the 'busybox' container,
the build completes successfully.
* If Jenkins is restarted, the build passes. But subsequent builds fail.
* The Agent logs contain similar exceptions whether the builds pass or
fail, so it's hard to pinpoint a cause.
The ThreadPool exception may imply that *something* has run out of
Executors, but it's not clear what, or how to rectify the situation.
Has anyone seen anything like this? Any ideas how to get to the bottom of
it? As per the plugin's README, I've created Jenkins log recorders for
org.csanchez.jenkins.plugins.kubernetes and okhttp3, but there are no
obvious problems reported in the logs.
Jenkins: v2.150.1
Kubernetes Plugin: 1.13.7
Kubernetes: 1.10.11 (same behaviour observed on 1.8.7)
All suggestions appreciated!
Thanks,
Simon.
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/141b526e-f00b-434d-9f43-ec97fdb23df7%40googlegroups.com
<https://groups.google.com/d/msgid/jenkinsci-users/141b526e-f00b-434d-9f43-ec97fdb23df7%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CALHFn6PM4RaDkYy-MZ8CJceoJxap-cxr3bMQZtR7oVzuJd2Tjg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Loading...