Once you've gone beyond a trivial number of Jenkins jobs you can get into a situation of not knowing which job does which thing. You might say "I know some job is running a query on a table, but which one?" in that case it can be helpful to search your Jenkins jobs.

Managing Jobs by script size

We have two strategies for managing Jenkins jobs: put short scripts into the job itself, but move longer jobs into code somewhere else.

Most jobs use the "execute shell command" Build Step. I also use and recommend the JobConfigHistory plugin to see how a job has changed over time (or who fixed/broke things). When the number of lines in your "shell script" section of the job gets over a maybe 3 or 4 I think it's time to move things to "real code" that is managed by a revision control system with more power than the JobConfigHistory plugin (i.e. git). So, we tend to put things into one of four places: R scripts, Pentaho jobs, Drush commands or bash scripts. All of the R scripts, drush commands and bash scripts are managed with git. The working checkouts of those directories are updated periodically (...by Jenkins jobs :)). So, if I want to grep that external code to see where a particular table is being modified it is very easy to do that. But...what about the one or two line shell scripts that are inside Jenkins jobs?

Searching Jenkins Job config files

First, you have to know about the Jenkins directory structure. The job configurations are stored in files called config.xml located in the Jenkins home directory (often /var/lib/jenkins/). So, if you have a job named production_deploy then the config file for it is located at /var/lib/jenkins/job/production_deploy/config.xml

If you want to grep all the config files to find which one is doing something with the field_revision_field_tags table you can use:


cd /var/lib/jenkins/jobs/
grep -Hni field_revision_field_tags */config.xml


• The H flag to grep tells you the name (including parent directory) of the file, which is handy in this case to know the name of the job.
• The n flag lets you know the line number the where the string occurs in the file. In case you want to open the config.xml directly with a text editor and go to that line number.
• The i flag for case insensitive just in case it matters and because usually there are few enough jobs that speed isn't a concern (insensitive searches are slower).

Finding overly complex Jenkins jobs

Now, let's say you like the idea of keeping jobs simple by moving complex jobs into external code. Here's a bonus command line invocation to find jobs that are getting a bit long:


wc -l */config.xml | sort -n | tail


The -l flag to the wc command will give the count of lines in each config.xml. Piping it to a numeric sort puts the longest jobs last and tail will show the longest 10 jobs. Investigating those will probably let you know which jobs need review for turning them into some other form of managed code.