"No one is harder on a talented person than the person themselves" - Linda Wilkinson ; "Trust your guts and don't follow the herd" ; "Validate direction not destination" ;

December 29, 2012

2012 Big Data Interesting Milestones

2012 was a fantastic year in terms of Big Data and Cloud Adoption. Massive Big Data focus and Products focussed on Big Data Analytics. Fortunately the world did not end as per Mayan Prediction.
 
How Traditional Players catch up in Big Data Space
  • Traditional RDBMS Vendors MSSQL, ORACLE offering their own NOSQL Solutions, Hadoop Connectors to leverage the power of Hadoop and enhance their BI offerings
  • IBM BigInsights Hadoop based Analytics Platform, Informatica and Microstrategy enhanced their Big Data Capabilities this year by providing integration with Hadoop Ecosystem
  • Acquire Hadoop based Startups to catch up with Big Analytics
Real Time Analytics Trend in 2012 
  • Cloudera launched Impala platform to provide Real Time Analytics
  • Real Time Analytics based on Twitter Storm
Big Data Fifth Elephant Conference provided good insight on Big Data Use Cases and Adoption rates

Big Data Products
Indian Startups Working on Big Data / Leveraging Hadoop Ecosystem for Analytics
  • Inmobi
  • Qubole
  • Flipkart
2013 Focus 
Interesting Big Data Reads

Inspiring Tweet to end the post



Happy Learning!!!

  

December 09, 2012

Python Basics

Two Coursera classes (An Introduction to Interactive Programming in Python, Learn to Program: The Fundamentals) provide very good python beginner learning materials. Python is a Interpreted Language

Quick Summary and Python Notes
Basic Examples and beginner notes posted based on course notes. All Code can be tried in CodeSkulptor.  There are a few syntax differences but this is similar to C programming language.

Example1 - Arithmetic Expressions
  • Basic Arithmetic Expressions
  • Variables, Assignments, Sample Exercise in link
All Scripts can be run in FireFox / Chrome in CodeSkulptor (Not in IE)



Example 2 - Working with Functions
Built In functions - max, abs
Example 3 - Custom Functions
  • Example function to sum up two numbers
Example 4 - If Statement - Conditons , Else If and Else Operator
Example 5 - For Loop - Iteration
Example 6 - While Loop iteration
Happy Learning!!!

December 01, 2012

Tool Developer Notes


Tip #1 - Sending Email on Error using Log4NET

Tip#2 - Append XML Node in C#

Before Code Execution


Post Code Execution



using System;
using System.Globalization;
using System.Data;
using System.IO;
using System.Xml;
namespace ExampleCode
{
    public class ExampleCode
    {
        static void Main()
        {

            //Open XML
            XmlDocument xmlPreCountFileData = new XmlDocument();
            xmlPreCountFileData.Load("E:\\abc.xml");
            XmlElement elmXML = xmlPreCountFileData.CreateElement("Node");

            //Append Node
            //Append Existing XML Node
            string xmlDataElement = @"<A>40</A>";

            elmXML.InnerXml = xmlDataElement;
            xmlPreCountFileData.DocumentElement.AppendChild(elmXML);
            xmlPreCountFileData.Save("E:\\abc.xml");
            Console.ReadLine();
        }
    }
}



Tip #3 - Thread Enhancements in .NET 4.0

Threading made easy in .NET 4.0
C# Mulththreading Improvements in .NET 4.0
.NET 4.0 and System.Threading.Tasks


Happy Learning!!!

November 16, 2012

Big Data Trends

Excellent Presentation on Big Data Trends..






Happy Reading!!!

October 28, 2012

Hadoop Single Server Setup on Ubuntu


In continuation with previous post we will install java, hadoop, configure map reduce jobs on ubuntu. The step by step details are based on details provided in link.

Many thanks to Arun and Pragalathan for helping me learn linux :)

Step 1 - Installing JDK



Step 2 - Create Hadoop Group


Step 3 - Create hduser


Step 4 - Add sshkey


Step 5 - Enable SSH


Step 6 - Install SSH (Had some issues so reinstalled it)

Step 7 - ssh to localhost



Step 8 - Disable IPV6 (Modify the entries based on steps provided in link)



Step 9 - Download Hadoop


Step 10 - Move it to Hadoop folder


Step 11 - Make permission owner hduser

Step 12 - bashrc changes


Step 13 - Check Java and Hadoop Home Path


Step 14 - Hadoop env changes


Step 15 - Modify custom xml files in below folder


Step 16 - Start Service

Step 17 - Copy files for working example


Step 18 - Hadoop file system details


Step 19 - Copy Files to Hadoop File System


Step 20 - List uploaded files


Step 21 - Run wordcount example jar


Step 22 - Check results output file


More ReadsHow to Install Hadoop on Ubuntu 13.10

Happy Learning!!!

Ubuntu Installation - Part I

Guideline for setup is based on Steps provided in link

Setting it on Ubuntu desktop, step by step details with Snapshot. Download installer from link
Step 1 – Installing Ubuntu


Step 2 – Ubuntu Installation

 
Step 3 – Drive to Install


Step 4 – Provide Credentials


Step 5 – Configure Network

Follow the next steps and complete installation and logon to system.

Happy Learning!!!

October 08, 2012

Interesting Post - Learn to Code (Startup Tips)

This post is to highlight few lines from Post on Learning to Code. If you have plans to do a startup it is useful and advantageous if you know to write code, passion for technology.



Happy Learning!!!

October 07, 2012

Algorithms Course Notes - Part II

Algorithms Course Notes from Coursera

Binary Search Trees
  • Binary Tree in Symmetric order (Nodes - Contain info, two links)
  • Each node has left and right tree, both can be null
  • Every Node Key Larger than keys in left sub tree
  • Every Node key smaller than keys in right sub tree
  • Inserts - Find a null link by search eligible position and insert it there
B Trees
  • General Model for external storage
  • Internal node key guide search
  • External node has client key
  • Insert at Bottom null node
  • If Nodes are full it will split to allow insert
  • Variants of BTree is used in DBs (B+ Tree, B* Tree)
RBT - Red Black Tree Tracks every simple path from a node to a descendant node with same number of black nodes
Red Black Trees
  • Internal Left Leaning links to glue 3 nodes
  • No Nodes have two red links connected
  • Every path from root to null link has same number of black links
  • Use a flag color to denote red or black link in implementation
Heap Sort
  • Largest of all keys is the root
Reposting Important Summary Slides

Happy Learning!!!

October 03, 2012

Jenkins Installation on Windows Server


This post is about issue encountered today and resolution for the same. Below error was returned by Jenkins windows installer


There was no log file associated with it. To debug it we need the log file for this installation. Below command would provide us log file


After few google searches enable .NET 3.5 features, this did not resolve the error

By closely examining the errors you would see error

ExecFirewallExceptions: Installing firewall exception2 Jenkins (C:
\Program Files (x86)\Jenkins\jre\bin\java.exe)

Fix was it was provided in link. By enabling windows firewall installation was successful.

Posting it hoping it is useful for larger audience.

Happy Learning!!!

September 22, 2012

Learning Ruby - Part I

In continuation with previous post Ruby Getting Started we will look at examples for loop, multiplication tables, File I/O operations. I am referring to Beginning Ruby by Peter Cooper for learning Ruby Programming Syntax

Example program multiple.rb

x = 1;
#Example 1 - Multiplication using 10.times;
puts "From 1 to 10 Multiplication of 5";
10.times do
       a = x*5;
       puts a;
       x = x+1;
end;
#Example 2 - Loop using upto;
puts "From 10 to 20 Multiplication of 5";
1.upto(10) do
       a = x*5;
       puts a;
       x = x+1;
end;
puts "Arrays"
#Example 3 - Arrays list all elements;
arrayA = [1,2,3,4,5];
arrayA.each {|y| puts y};
puts "Array & Strings"
arrayB = [1,2,"One","Two","Three"];
arrayB.each {|y| puts y};
#Example 4 - List the program line by Line;
File.open("d:\multiply.rb").each { |line| puts line }

 
 
Happy Learning!!! 

September 09, 2012

C# Fundamentals


This time is all about fundamentals of C# . Going beyond theoretical basics.

Tip #1 - Why Value Types are stored in Stack and not in Heap
This blog post (The Truth About Value Types) was the answer. It is a recommendation that CLR does on your behalf to store value types in stack. Short Lived storage (value types) live in stack. Long duration (reference types) live in heap

Tip #2 - Garbage Collection process, Phases Involved
Mark, Sweep and Compact. More details refer The Stack Is An Implementation Detail, Part Two

Tip #3 - Abstract Class VS Interfaces
  • Interface - Can define methods but not implement them
  • Abstarct Class - Can define/implement methods/members but cannot instantiate abstract classes
using System;
namespace examplecode
{
    abstract class AClass
    {
        int aMember;
        public void printVal()
        {
            Console.WriteLine("Example AbstractClass");
        }
    }
    interface iTest
    {
        void printValInterface();
    }
    class Program:AClass,iTest
    {
        public void printValInterface()
        {
            Console.WriteLine("Example printValInterface");
        }
        public static void Main()
        {
            Program RunProgram = new Program();
            RunProgram.printVal();
            RunProgram.printValInterface();
            Console.ReadLine();
        }
    }
}
 
Tip #4 - Why multiple inheritance is not supported in .NET
Answer from stackoverflow question (Why is Multiple Inheritance not allowed in Java or C#?) is realistic and impressive. The benefits are too less and adds a lot of complexity to support multiple inheritance.
 
Tip #5 - Disassembly Good Example
 
Couple of MSDN blog posts were very very impressive. A must read for every developer
 
Happy Learning!!!

September 04, 2012

Ruby Getting Started


This post is about learning ruby. Tried it online using rubyfiddle. rubbyfiddle did not work with global variables i.e Variables using $ symbol.

Basic Commands

Very Usual Hello World Example


Dynamic Typing



Global variables didn't work using ruby fiddle. Installed ruby for Windows. Using Interactive Ruby Option under Programs->Ruby tried below examples

While Loop


Next example is trying out for loop. Below is the test file


Start Command Prompt with Ruby. Run the test file


Pretty easy to learn. Looking forward for more interesting posts.

More Reads
Ruby Tutorial with Code Samples
A Wealth Of Ruby Loops And Iterators
Ruby Procs And Lambdas (And The Difference Between Them)
A Unit Testing Framework In 44 Lines Of Ruby


Happy Learning!!!

August 30, 2012

Algorithm Analysis - Basics

This post is based on notes after attending Algorithms I classes at Coursera. Lecture on Analysis of Algorithms is very good. Couple of interesting snapshots/ questions captured during the course posted below


This can be computed to 1000000 / log (1000000) = 100000/6 = 1,66,6666. This should be the right answer I believe.

Another interesting slide - Order of Growth




  • Big-O is an upper bound.
  • Big-Theta is a tight bound, i.e. upper and lower bound.Link

Another Interesting Question

 
 
 
Happy Learning!!!