REST load testing with siege

Siege is a pretty nifty linux utility for HTTP/HTTPS load testing it can be downloaded here : siege homepage
Siege allows you to write a list of URLs in a file (one URL per line) the utility will then parse the file and execute the load tests according to your configuration file .siegerc or the command's arguments.

The only thing I do not like about siege is the fact that I would like to have stats per URL; when siege parses the url file it will write a log file with global statics for the siege session

Something like so :

2012-09-21 12:03:52,   1813,     121.43,          19,       0.49,       14.93,        0.16,        7.38,    1813,       0

which to my taste is not very clear

While executing the tests siege prints out statistics that are not found in the siege log file (which in my case are more useful) but if you have multiple URLs you don't know for which URL the printed stats are.

So I decided to handle the situation differently using awk and a bash script

Note that I'm no Linux expert so the script could probably be better written but here is how I broke it down :

I have 3 files

1. The urls.txt file (a txt file containing the URLs I want to test :

#URL1
http://myURL1/param1/param2.json
http://myURL1/param3/param2.json
#URL2
http://myURL2/param3/param2.json
http://myURL2/param3/param1.json

2. The AWK script file

!/#/{

   print "\n"
   print "**********************************************************"
   print "Testing URL : "$1
   print "**********************************************************"
   system ("echo Testing URL : "$1 " >> " SIEGE_OUTPUT " 2>&1")
   system ("siege " $1 "-v -b -r"REPS" -c"CONC" --mark="$1 " --log="SIEGE_LOG_FILE ">> "SIEGE_OUTPUT " 2>&1 ")
   print "\n"

}

3. The bash script containing all the configuration and calling the awk script

#! /bin/sh
#LOG_FILE=./siege-log
URL_FILE=./urls.txt # file containing the URLs to handle
USER=$(whoami)
NOW=$(date +"%d-%m-%Y_%H-%M-%S")
PWD=$(pwd)
SIEGE_LOG_FILE=$PWD"/log_"$NOW".log"  #siege log's file
SIEGE_OUTPUT=$PWD"/output_"$NOW".log" #where siege's output will be redirected
CONC=50 #number of concurrent users
REPS=1  # number of repetitions
echo "loading URLS from the file : "$URL_FILE
echo "Writing siege log into the file :" $SIEGE_LOG_FILE

SET -- $CONC

awk -f siege-benchmark.awk -v CONC=$CONC REPS=$REPS SIEGE_LOG_FILE=$SIEGE_LOG_FILE SIEGE_OUTPUT=$SIEGE_OUTPUT $URL_FILE

Now according to how your siegerc file is configured output can be a bit different I have the verbose mod off and benchmark mode on. Below is an excerpt from my log file

******************************************
Testing URL : http://myURL1/param1/param2.json
******************************************
** SIEGE 2.72
** Preparing 50 concurrent users for battle.
The server is now under siege...


Transactions:                     50 hits
Availability:                 100.00 %
Elapsed time:                   0.08 secs
Data transferred:               0.00 MB
Response time:                  0.04 secs
Transaction rate:             625.00 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                   24.62
Successful transactions:          50
Failed transactions:               0
Longest transaction:            0.07
Shortest transaction:           0.01

MongoDB & Spring Data remove Elements from Arrays

In this blogpost I will be showing how to execute a Mongo Query using Spring Data in order to "pull" multiple items in a object's collection. The model :

@Document(collection="Book")
 public class Book{

      private String ISBN;
      private String bookTitle;
      private List< String > categories;


  //SETTERS - GETTERS
  }

  public class Category{
    
      private String categoryId;
      private String categoryName;
   }

A typical Mongo Document would be something like :

   {
      "isbn" : "11111111", 
      "bookTitle" : "My Book Title", 
      "categories" :[
          {
            "categoryId" : "cat1",
            "categoryName": "My Category 1"
          },
          {
            "categoryId" : "cat2",
            "categoryName": "My Category 2"
          }
        ]
     }

    {
      "isbn" : "22222222222", 
      "bookTitle" : "My Book Title2", 
      "categories" :[
          {
            "categoryId" : "cat1",
            "categoryName": "My Category 1"
          },
          {
            "categoryId" : "cat3",
            "categoryName": "My Category 3"
          }
        ]
     }

     {
      "isbn" : "333333333", 
      "bookTitle" : "My Book Title3", 
      "categories" :[
          {
            "categoryId" : "cat2",
            "categoryName": "My Category 2"
          },
          {
            "categoryId" : "cat3",
            "categoryName": "My Category 3"
          }
        ]
     }

Now let's assume that cat1 and cat2 are no longer valid. It would be nice if we could remove these categories from all the books in one mongo update query. Here is how to do it using Spring Data.

@Component
public class MyDAO{
   
   // Mongo template injected through Spring

   @Autowired
   private MongoOperations mongoTemplate;

    public void deleteCategories(String[] categories){
     
      //find query
      // find all the programs containing the the categories passed as parameter
      Query findQuery = 
       Query.query(Criteria.where("categories.categoryId").in(Arrays.asList(categories)));      


       // build the Update
        DBObject pullUpdate = 
            BasicDBObjectBuilder.start().add(
               "categoryId",
                      BasicDBObjectBuilder.start()
                                              .add("$in",
                            categories).get()).get();
    
      Update update = new Update().pull("categories", pullUpdate );
     

     // execute the update
      template.updateMulti(findQuery, update, Book.class);

   }

}

    public class SpringDataDemo{
      
       public static void main(String[] args){

             ClassPathXmlApplicationContext context = 
                  new ClassPathXmlApplicationContext("classpath:spring-beans.xml");
             MyDAO myDao = context.getBean("myDao");
             //delete categories "cat1" and "cat2"
             mydao.deleteCategories(new String[]{"cat1", "cat2"});
       
       }
    }

Now the categories cat1 and cat2 will be removed from all the books in the collection.