Jenkins - Setup to schedule your repetitive scripts

What is Jenkins & when to use It?

When you have bunch of repeated jobs or scripts that you want to run, scheduled or sometimes manually triggered. Go for jenkins.

Its as easy as 

•    Step 1: download
•    Step 2: Install
•    Step 3: Install Basic Plugins (optional)
•    Step 4: Setup security
•    Step 5: Create new job(s)
•    Run

Jenkins is also used these days in the case of continuous integration, working with the Source Control, Review Boards, and other quality monitoring tools.

Step 1 : Download  : http://jenkins-ci.org/

Based on your environment, you can download the installer as shown below and install it.

Step 2 : Install   - Follow instructions.

• Follow the installer instructions.
• On Linux, firewall to be disabled if it blocks 8080 port
  service iptables off
• After installation accessible at  http://<hostname>:8080  . Example http://localhost:8080

You will see the below screen after successful installation & start-up

Step 3 : Install  Plugins (Optional)

There are some default plugins. If you need more, pick and choose from the plugin list as shown below.

For Manual installation.

• Download plugins from https://updates.jenkins-ci.org/download/plugins/
• copy the *.hpi/*.jpi file into the $JENKINS_HOME/plugins directory. 
• Restart Jenkins (many containers let you do this without restarting the container)

I picked up few extra plugins Greenballs, XShell, setenv

Step 4 : Enable Security

1. Manage Jenkins -> Configure Global Security -> Enable Security.  

2. Configure as shown in the below picture.

1. Enable Security
2. Save  in jenkins database with Signup Enabled. This will enable you to add users later.
3. Add privileges for admin user (which you will add through signup process)
4. anonymous will have no privileges. Admin user - Select all checkboxes

1. 5. Save, which will take you to the Login page as shown in the next image.

3. Signup for admin user from the Login Page. 

You are done with the basic security. You can go ahead and add other privileges using the admin user that is created.

Step 5 : Create new job

The dashboard now shows the job that was created.  and can now be run to see the progress

Some special settings when running on Linux Environment.

Jenkins in linux is created as a service and can be started using service jenkins start

The service will not be able access scripts or files within /root user  Say java -jar /root/test.jar will give you permission denied, even after setting the chmod appropriately. 

To solve this problem, the sudoers file has to be edited as mentioned below and execute the command prefixed with sudo. 

sudo java -jar /root/test.jar

Changes in the sudoers file

• requiretty to be disabled for the jenkins service.  Add the Defaults:jenkins !requiretty  afterDefaults requiretty
• visudo .(edit sudoers files)
• Defaults requiretty
• Defaults:jenkins !requiretty
• add this line jenkins        ALL=(ALL)       NOPASSWD: ALL at the end

[Solution] Elasticsearch 0.90 insallation with .rpm and logging Errors

Problem : 

I downloaded the latest ES rpm and installed on centos. When tried running the script in foreground, i see that the logging is not enabled.

tried both /etc/init.d/elasticsearch -f and   /usr/share/elasticsearch/bin/ -f both. 

log4j:WARN No appenders could be found for logger (node).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

and nothing gets logged.

Tried the following in vain : 

• export ES_CLASSPATH=/etc/elasticsearch/logging.yml 
• changed patch.conf, path.data in elasticsearch.yml file

But, Nothing seems to fix the logging problem

Solution:

edit /usr/share/elasticsearch/bin/elasticsearch  

give absolute paths for the home, and all files.

This file should be changed, so "any" startup of elasticsearch will work in foreground without errors

ES_JAVA_OPTS="-Des.config=/etc/elasticsearch/elasticsearch.yml -Des.path.conf=/etc/elasticsearch/ -Des.path.home=/usr/share/elasticsearch -Des.path.logs=/var/log/elasticsearch -Des.path.data=/var/lib/elasticsearch -Des.path.work=/tmp/elasticsearch -Des.path.plugins=/usr/share/elasticsearch/plugins"

[Book Review] ElasticSearch Server for fast scalable flexible search solution

ElasticSearch Server 

http://www.packtpub.com/elasticsearch-server-for-fast-scalable-flexible-search-solution/book

This is a good book for someone who wants understand and try out the latest search framework ElasticSearch built on top of Lucene. This claims to provide the best, flexible, distributed and enterprise Search Framework

It covers almost all the topic that is required

1. Introduction to Elastic Search
2. Understanding Shards, Nodes and cluster.
3. Indexing and Searching Techniques
4. Templates, Mapping
5. Different API's
6. Plugins.
7. etc.,

I would suggest this book for someone interested in ElasticSearch Server.

Java: Code to unquote string using Regex

str="\"hello\"" // or "'hello'"

public static unquote(String str)
{
        str = str.trim();
if (str.startsWith("'") && str.endsWith("'"))
{
    str = match(str, "(?<=').*(?=')");
}
else if (str.startsWith("\"") && str.endsWith("\""))
{
         str = match(str, "(?<=\").*(?=\")");
}
}

//matches the pattern
private static String match(final String string, final String regex)
{
final Pattern pattern = Pattern.compile(regex);
final Matcher matcher = pattern.matcher(string);
matcher.find();
return matcher.group(); 
}

[Shared Webinar] Getting Started with ElasticSearch

Click on the Below Image and fill in details to watch the Webinar

Quick Notes : Practical usage of collections

Different Types of Collections and when to Use them

Set

HashSet : 

• When you want unique set to objects. 
• Make sure equals and hashcode is implemented.
• This does not maintain the order of object addition. As the name says it is Hashed

TreeSet:

• You can have the list ordered the way you want, by passing in a comparator.
• Example: new TreeSet(String.CASE_INSENSITIVE_ORDER)
• It automatically orders the objects that you add accordingly

LinkedHashSet

• This gives us the flexibility of HashSet (uniqueness) and 
• also maintains the insertion-order of objects

Map

HashMap :

• Easier way of holding Key Value Pair
• Does not maintain the insertion-order (Hashed)

TreeMap:

• Sorted Map, Sorting of keys determined by the comparator in the constructor
• Quickly create a map of case insensitive keys as follows
  new TreeMap(String.CASE_INSENSITIVE_ORDER)
• Throws a NullPointerException if you use get without a null check.
  treemap.get(null), so better add a null check before invoking get()

LinkedHashMap:

• Extends HashMap. But, preserves of the insertion order

List:

ArrayList:

• Quicker way of maintaining a list of objects
• can add duplicate values.
• It is advisable to use this collection if we expect more number of gets() than add() or set()

LinkedList:

• Performs better when you have more additions and insertions than searching the list

Deleting Public Keys from GPG and RPM key rings

In order to delete the public keys from the key ring, we would require the key_id. The key_id can be idenfied using the below steps using the file where the public key is stored.

To get the Key Details

gpg --throw-keyids < [Key-File-Absolute Path]

pub  1021D/1689B887 2005-03-11 ABC Company 

sub  1048f/47E5X96D 2005-03-11  [expires: 2015-03-09]

To get the Key-ID from the keyfile

key_id=${gpg --throw-keyids < [Key-File-Absolute Path] | head -1 |  awk {'print $2'} |  cut -d '/' -f2 |  tr [A-Z] [a-z]}

1689B887

Delete from GPG keyring

gpg --delete-key  --yes $key_id

Delete from RPM keyring

rpm -e --allmatches gpg-pubkey-$key_id