Mihamina Rakotomandimby

To content | To menu | To search

Thursday 25 August 2011

ocaml random string and word generation

Goal

I had to insert sample data in a database:

  • Users
    • username
    • enabled
  • Products
    • name
    • description
    • price
All should be random, and to "benchmark" the rendering (HTML/CSS), I must make the effort to have different text and number length. I decide to generate the sample data with OCaml.

Random Int

Generate a random integer between 1 and 1000 and return it as a string:

        let rand_price () = string_of_int (1+ (Random.int 999)) ;;
      

Random Char

Generate a random lowercase character (between 'a' and 'z'). 'a' ASCII code is 97 and the 26 following letters are the 26 following numbers:

    
        let rand_chr () = (Char.chr (97 + (Random.int 26)));;
      

Random vowel

Generate a random character until it matches a vowel. Please add a comment if you would suggest another algorithm:

    
        let rec rand_voy () = 
        let got = (rand_chr ())
          in 
          match got with 
            | 'a' | 'e' | 'i' | 'o' | 'u' | 'y' ->  got 
            | _ -> rand_voy ();;
      

Random consonant

Generate a random character until it doesnt matche a vowel. Please add a comment if you would suggest another algorithm:

        let rec rand_con () = 
          let got = (rand_chr ())
            in 
            match got with 
              | 'a' | 'e' | 'i' | 'o' | 'u' | 'y' ->  rand_con ()
              | _ -> got ;;
      

Random syllable(s)

In this document, a syllable is a consonant followed by a vowel. Generate a finite number of random syllables:

        let rec rand_convoy acc syll_number () =
          match syll_number with 
            | 0 -> acc;
            | _ -> rand_convoy (acc ^ (Char.escaped (rand_con ())) ^ (Char.escaped(rand_voy()))) (syll_number - 1) ();;
      
The final goal is to use all this as a "string", so we make "char" to "string" conversion at this level with the "Char.escaped" function.

Random short word

Short word: between 3 and 6 syllables.

        (* mot: entre 3 et 6 syllabes *)
        let rand_word () = (rand_convoy "" (3 + (Random.int 3)) ());;
      
Wel will also need a fixed length word, of 4 syllables.
 
        (* nom: 4 syllabes c'est tout *)
        let rand_name () = rand_convoy "" 3 ();;
      

Random sentence

Random sentence is a random number of random words. We will limit the number of words (to avoid bloating the database). Wel will end the sentences with a ".".

        let rec rand_sentence acc word_number () =
          match word_number with 
            | 0 ->               (acc ^ (rand_word ()) ^ ".");
            | _ -> rand_sentence (acc ^ (rand_word ()) ^ " ") (word_number - 1) ();;
      

Utilities

Generate a "description", and quote a string in preparation for INSERT in the database:

        let rand_description () = rand_sentence "" (10 + (Random.int 10))  ();;
        let sql_quote a_string = "'"^a_string^"'";;
        let generate_insert_user () = "INSERT INTO users VALUES ("^sql_quote (rand_name ())^" ,1)" ;;
        let generate_insert_product p_id  = 
          "INSERT INTO products VALUES ("^(string_of_int p_id)^", '"^(rand_name ())^"', '"^(rand_description ())^"', "^(rand_price ())^")";;
      

SQLite filling

SQLite manipulation has already been introduced in another post.

        let db = Sqlite3.db_open "/var/www/database.db";;

        Sqlite3.exec db (generate_insert_user ());;
        let rec fill_users number = 
          match number with 
            | 0 -> ()
            | _ -> 
              Sqlite3.exec db (generate_insert_user ()) ; 
              fill_users (number-1); 
              ();;
        
        let rec fill_products number =  
          match number with 
            | 0 -> ()
            | _ -> 
              Sqlite3.exec db (generate_insert_product number) ; 
              fill_products (number-1); 
              ();;
      

Wednesday 17 August 2011

ocaml sqlite3 date

Use case

I have a SQLite3 database, with Date, Integers and Float fields. I use it to make a poor-man accountig of my ppp0 interface traffic. Data I want to play with are in the format:

$ sqlite3 accouting-copy "SELECT * FROM ifconfig"
1|2011-08-12 09:37:47|0.0       |0.0
2|2011-08-12 09:43:01|18629153.0|8124895.0
3|2011-08-12 09:43:03|18636044.0|8125679.0
4|2011-08-12 09:44:32|18694283.0|8159197.0
5|2011-08-12 09:50:01|19203494.0|8270963.0
6|2011-08-12 09:55:01|19265098.0|8311962.0
Where:
  • Field #1: integer, incremental
  • Field #2: date, SQLite "DATETIME('NOW')"
  • Field #3 and #4: Float, "RX" and "TX" counters parsed from "ifconfig ppp0". Sometimes, these counters reset: If the modem hangs and I must perform a "ifdown ppp0; ifup ppp0". That is why I need computation.

I want to account the traffic:

  • Between to Dates: To know how much I had for a given month
  • From a given Date to Now(): To approximately know my remaining traffic

I am not going to care about to computing logic in this article, but focus on data extraction.

Preparing the toplevel

In order to use these examples, those modules are needed:

#use "topfind";;
#require "calendar";;
#require "sqlite3";;

Auxilliary functions and variables

Some variables:

let db = Sqlite3.db_open "/home/mihamina/accouting-copy";;
let the_query = "SELECT * FROM ifconfig";;

I need a couple of auxilliary functions.

To convert a "String Option" (Some "foo") to a String:

let so_to_strig the_so =
match the_so with
| Some s -> s;
| _ -> "";;

To get a Calendar Date from a an SQLite Date String:

let date_from_sql  = CalendarLib.Printer.Precise_Fcalendar.from_fstring "%F %T" ;;

To print a Date:

let date_to_string = CalendarLib.Printer.Precise_Fcalendar.to_string ;;

To add tso days to a Date (For testing or demonstration prupose):

let add_2_days a_date = CalendarLib.Fcalendar.Precise.add a_date (CalendarLib.Fcalendar.Precise.Period.make 0 0 2 0 0 0.);;

The callback just to print the table content:

let the_print_callback row headers =
(Array.iter (fun s -> Printf.printf "  %-12s"              s)  headers);
(print_endline "");
(Array.iter (fun s -> Printf.printf "  %-12s" (so_to_strig s)) row);
(print_endline "");;
Thanks and Acknowledgments

The callback to play with the data, which is the most important for the work:

let the_data_callback row headers=
(* row.(0) row.(1)  row.(2)  row.(3)  *)
(* id      date     rx       tx       *)
(* int     Date     float    float    *)
(* "headers" is no used at the moment *)
let the_id = int_of_string   (so_to_strig (row.(0)))
and the_rx = float_of_string (so_to_strig (row.(2)))
and the_tx = float_of_string (so_to_strig (row.(3)))
and the_date = CalendarLib.Printer.Precise_Fcalendar.from_fstring "%F %T" (so_to_strig (row.(1)))
in
print_string "the_id: "  ; print_int   the_id                    ; print_string "\t\t the_id doubled: "        ; print_int    (the_id * 2)                          ; print_string "\n";
print_string "the_tx: "  ; print_float the_tx                    ; print_string "\t\t the_tx doubled: "        ; print_float  (the_tx *. 2.)                        ; print_string "\n";
print_string "the_rx: "  ; print_float the_rx                    ; print_string "\t\t the_rx doubled: "        ; print_float  (the_rx *. 2.)                        ; print_string "\n";
print_string "the_date: "; print_string (date_to_string the_date); print_string "\t\t twodays after the_date: "; print_string (date_to_string (add_2_days the_date)); print_string "\n";
print_endline "==================================================================================================================";
;;

Launching

To print all the table content:

let result = Sqlite3.exec db ~cb:the_print_callback the_query;;

To make sample operations with the data:

  • double the ID and print it
  • double the RX and TX and print them
  • add 2 days to the Date and print it
let result = Sqlite3.exec db ~cb:the_data_callback the_query;;

Sample Source code

The source code is on my Google Code Repository

more on precise date on ocaml

Dealing with "Precise" Dates

I already posted an article on (begining) playing with Dates on OCaml.

Preparing the Toplevel

As usual, we have to load and use several modules:

#use "topfind";;
#require "calendar";;

Creating a Second

To create a second, the top reachable precision:

CalendarLib.Time.Second.from_int 5;;

Creating a Precise Date

To create a precise date, let's use the "make" function:

CalendarLib.Fcalendar.Precise.make 2011 08 12 05 32 (float_of_int (CalendarLib.Time.Second.from_int 5));;
CalendarLib.Fcalendar.Precise.make 2011 08 12 05 32 5.;;

Create a Precise Period

When wanting to add 2 days to a date, we must use "The Date + 2 days Period". To create that "Period":

CalendarLib.Fcalendar.Precise.Period.make 2011 08 12 05 32 (float_of_int (CalendarLib.Time.Second.from_int 5));;
CalendarLib.Fcalendar.Precise.Period.make 2011 08 12 05 32 5.;;

Make an operation (add)

Let's create a Date, and calculate the "two days later" Date:

let d2 = CalendarLib.Fcalendar.Precise.add
(CalendarLib.Fcalendar.Precise.make 2011 08 12 05 32 (float_of_int (CalendarLib.Time.Second.from_int 5)))
(CalendarLib.Fcalendar.Precise.Period.make 0 0 2 0 0 (float_of_int (CalendarLib.Time.Second.from_int 0)))
;;
let d2 = CalendarLib.Fcalendar.Precise.add
(CalendarLib.Fcalendar.Precise.make 2011 08 12 05 32 5.)
(CalendarLib.Fcalendar.Precise.Period.make 0 0 2 0 0 0.)
;;
Then display it:
CalendarLib.Printer.Precise_Fcalendar.to_string d2;;

If the Date was from a string

I mostly intend to take the Date from an external source: A "date" field from a SQLite3 database, which is formatted "2011-08-16 16:00:01". To use it:

let initiale = CalendarLib.Printer.Precise_Fcalendar.from_fstring "%F %T" "2011-08-16 16:00:01";;
Then to add 2 days to it:
let initiale_2 =
CalendarLib.Fcalendar.Precise.add
initiale
(CalendarLib.Fcalendar.Precise.Period.make 0 0 2 0 0 (float_of_int (CalendarLib.Time.Second.from_int 0)))
;;
Finally print it:
CalendarLib.Printer.Precise_Fcalendar.to_string initiale_2;;

Tuesday 16 August 2011

tutorial date ocaml

OCaml using Calendar

Some acknowledgement

From the toplevel

# #use "topfind";;
# #require "calendar";;
/usr/lib/ocaml/unix.cma: loaded
/usr/lib/ocaml/str.cma: loaded
/usr/lib/ocaml/calendar: added to search path
/usr/lib/ocaml/calendar/calendarLib.cma: loaded

Note that the library is "Calendar", but "calendarLib.cma" is loaded. So, to set a Date from a String:

# let d = CalendarLib.Printer.Precise_Fcalendar.from_fstring "%F %T" "2011-08-16 16:00:01";;
val d : CalendarLib.Printer.Precise_Fcalendar.t = 
Or to print a Date to a String:
# CalendarLib.Printer.Precise_Fcalendar.to_string d;;
- : string = "2011-08-16 16:00:01"
# CalendarLib.Printer.Precise_Fcalendar.sprint "%Y-%m-%d %H" d;;
- : string = "2011-08-16 16"
# CalendarLib.Printer.Precise_Fcalendar.sprint "%Y-%m-%d %H:%M:%S" d;;
- : string = "2011-08-16 16:00:01"

Wednesday 20 July 2011

OCaml RSS Feed Parser

Using Cameleon RSS Feed Parser

Looking for a way to parse RSS (or ATOM) feeds with OCaml, I performed a Google search on "ocaml rss":

Here is how to begin to use it...

Installing on Ubuntu/Debian

I assume the user has all de basic OCaml installation. Installing the RSS feed parser requires:

    sudo apt-get install libcameleon-ocaml-dev cameleon
  

Getting a sample RSS file

Let's get the RSS2 file of OCamlCore:

    wget http://planet.ocamlcore.org/rss20.xml
  
The file is saved as "rss20.xml" use the -O option of wget if you want another saving name.

Using it in the toplevel

The user needs to load several modules:

       Objective Caml version 3.11.2


    #use "topfind"                  ;;
    #require "unix"                 ;;
    #require "cameleon"             ;;
    #require "cameleon.config_file" ;;
    #require "cameleon.configwin"   ;;
    #require "cameleon.ffind"       ;;
    #require "cameleon.gdir"        ;;
    #require "cameleon.gmylist"     ;;
    #require "cameleon.gstuff"      ;;
    #require "cameleon.okey"        ;;
    #require "cameleon.report"      ;;
    #load "/usr/lib/ocaml/cameleon/rss.cma";;
  
Load the XML RSS file (Pretty Long output)
    let my_channel = Rss.channel_of_file "rss20.xml";;
  
Get the list of items (Pretty Long output)
    let items = my_channel.Rss.ch_items);;
  
Grab the first item
    List.hd items;;
  
Get the title of that first item
    (List.hd items).Rss.item_title;;
  

Note

Please note that it is a quick and dirty way to do it, the reader is encouraged to suggest cleaner ways by commenting this post.

- page 1 of 4