"No one is harder on a talented person than the person themselves" - Linda Wilkinson ; "Trust your guts and don't follow the herd" ; "Validate direction not destination" ;

December 15, 2013

Weekend Reading Notes

Note #1 - SQL Server Database Engine Performance Tuning Basics - Must read for every TSQL Developer
Key Learning's
  • Perf counter values analysis
  • Temp Db configuration
  • Enable Lock pages in memory
  • Interpreting Avg. Disk Sec/Read values

Key Query  - SELECT SERVERPROPERTY('productversion'), SERVERPROPERTY ('productlevel'), SERVERPROPERTY ('edition')

Note #3 - One more NOSQL DB that supports ACID transactions - FoundationDB

Key Learning's
  • Memory optimized Tables(MOT) reside in memory not in disks
  • Steps to estimate memory for MOT
  • Garbage collection of older version of records (similar to snapshot / read committed isolation levels)

Happy Learning!!!

December 12, 2013

DataScience Basics - Part I

Welcome to series of posts to learn Data Science basics. This video is a good starting material to get started



Link

Key Learnings
  • Data Science term defined by Peter Naur
  • How Role of Statistician differs from Data Scientist
  • BI Tools vs Data Science perspectives
More Reads
Happy Reading!!!

December 11, 2013

Getting Started with Python Visualization

This post is about getting started with visualization using python. This took less than an hour to see visual data representation

Below are the steps involved for our first example

Step 1 - Install Enthought Python Distribution (EPD) from link. Download was around 230MB, This took a couple of minutes

Step 2 - This post was pretty useful on installation on Windows

Step 3 - Post Installation below setting was done

Step 4 - The first example is from python book page 37

Step 5 - Create a new file, type the code, Save and Run it

Step 6 - Below is the output 

December 05, 2013

Create Dummy Files to consume diskspace - fsutil command

This post is about creating dummy files to consume disk space and mimic scenarios with reduced disk space. On windows server 2008 fsutil command was useful to create files to consume disk space

Example Usage

C:\Users\Administrator>fsutil file createnew c:\dummydata5.bat 1240171806

File c:\dummydata5.bat is created

Happy Learning!!!


November 09, 2013

Weekend Reading Notes

Note #1 - Session - Dirty Truth about Data Literacy

Interesting notes captured during session

Data Literacy - In simple terms
  • Reading the Data (Understand Values)
  • Reading between Data (Comparisons)
  • Reading beyond Data (Predictions / Inferences)
Improving User Experience by 
  • More Confident Chart Readers
  • Interactive through roll over, tool tip, highlight selected numbers, mirror what they are doing 
Note #2 - Deploying Hadoop ETL in the Hortonworks Sandbox

This session is about working with SyncSort DMX-h tool to implement ETL. With the GUI interface you can define the jobs. The concept is something similar how you would develop using SSIS. Probably Microsoft also may come up with similar SSIS capabilities to run on hadoop and load data.

Syncsort ETL offerings link

Exercise #3 - Tutorial 14: How To Analyze Machine and Sensor Data

Steps were pretty easy, I was able to execute till step 5. Since I do not have office 2013 the GUI part is pending. Pretty simple and fast.

Interesting Reads
Hardware Considerations for In-Memory OLTP in SQL Server 2014
In-Memory OLTP Q & A: Myths and Realities
In-Memory OLTP: High Availability for Databases with Memory-Optimized Tables

Happy Learning!!!

September 29, 2013

Exploring Hortonworks Sandbox - Part I (on Windows 7)

Setup Steps
  1. Downloaded Virtual Box from Link
  2. Howtonworks windows tutorial Link
  3. Download 1.8GB Hortonworks Sandbox from Link 
  4. After Configuring it, Ran through the first tutorial - Link
  5. Started the Server and Open-up in browser IP address http://192.168.56.101:8000/ from Win7 Machine
  6. Sandbox was setup and configured to use 192 series IP Address. Was able to use the Win7 browser interface to perform file upload, query operations
  7. Credentials to logon on Server Login: root, Password: hadoop
  8. Command to shutdown is poweroff

First Example Notes
  • Downloaded example data from Link 
  • Upload worked in Google Chrome not in IE
  • Poweroff is the command to poweroff the sandbox machine
  • Uploading data, running basic Select queries worked fine
More Info Tutorials
My Feedback
  • Impressive easy setup and easy to use
  • Got Started in < 2 hrs
  • Good Learning Start
More Reads

WTF does a Data Scientist do all day?

How do I become a data scientist?

What are some software and skills that every Data Scientist should know?

Read Quote of Joe Blitzstein's answer to Data Science: What is it like to design a data science class? on Quora

Read Quote of Nishant Neeraj's answer to Big Data: What should be ideal size, skill set and composition of team for a successful Big Data implementation in an organization? on Quora

Read Quote of Sean Owen's answer to Job Interviews: How can a computer science graduate student prepare himself for data scientist/machine learning intern interviews? on Quora

Read Quote of Pronojit Saha's answer to Data Science: What are some software and skills that every Data Scientist should know? on Quora

Read Quote of Ye Zhao's answer to How do I become a data scientist? on Quora

How does one begin to learn data science?

Harvard Data Science Course  

Software engineer's guide to getting started with data science


Happy Learning!!!

September 20, 2013

Advanced Cloud Computing 2013 Notes


Advanced Cloud Computing 2013 Notes. Yesterday I attended ACC2013 held @ Nimhans. Every conference provides a lot of inspiration and motivation to try out new things.

Session #1 - Inauguration Talk

Inauguration was done by Padma Bushan  Rajaram. He explained cloud computing in very simple terms. Cloud computing is utility computing in simple terms. It was coined by a management professor named Chellapa.
He had earlier written a paper in 2005 on challenges in utility computing. He recollected his experiences with new jargon's/ technical abbreviations. He provided several examples on BYOD keyword. BYOD - Bring Your Own Drinks, BYOD - Bring Your Own Dope, BYOD - Bring Your Own Device (Recent usage). Storage and processing costs have come down. This has become a business potential for Amazon and Google to leverage it by outsourcing their excess storage and processing infrastructure.

He stressed on several areas to standardize the cloud for leverage the complete potential of it. Example- SLA to provide the required performance while hosting / sharing the infrastructure, Developing a universally usable cloud, interoperability between cloud providers

Session #2 - This was Talk by Karanataka's IT secretary VidyaShankar.IAS

He mentioned on developing trends Virtualization, Cloud and 3D Technologies. He mentioned
couple of products cloudmagic, cubby.

Session #3 - Connected Systems, Cloud beginner tech talk by Vikas Agarwal (Tally)

This talk pretty much focused on evolution of cloud computing. He tracked from the very beginning PC Era to cloud computing.
  •  Stage 1 - Mainframe Systems
  •  Stage 2 - PC Era (Moving data to personal systems)
  •  Stage 3 - LAN (Locally connected systems) - Intranets
  •  Stage 4 - Connected Era, WANs
  •  Stage 4 - Evolution of Internet (Globally Connected)
  •  Stage 5 - Cloud (Shared computing, storage) - Access anytime / anywhere
Challenges / Features in Cloud
  •  Pooling Optimization
  •  Elasticity
  •  Efficiency
Session #4 - Big Data in Safety & Security Domain, Tech talk by Bob Brewin (Tyco)

This talk focused on basics of cloud computing, challenges and applications in Fire and Security Domain. Key notes covered were
  • Fallacies in Distributed computing
  • Current Challenges in Fire & Security Domain are
  • Identifying False Positives
  • Predictive Analytics to identify and isolate false positives
  • Real time monitoring
Session #5 - Cloud Services in Yahoo by Jothi Padmanabhan

Yahoo has its own private cloud, Author provided details on Yahoo infrastructure and their software stack
 Challenges
  • Scaling systems as per growing data
  • Data Partitioning
  • Data Consistency
  • Hardware provisioning
Benefits of Private Cloud
  • Developers can focus on Application logic instead of designing for crash / recovery scenarios
  • Focus on appealing content for users (UX Exp)
Requirements for Cloud
Multitenancy
  • Several applications will share the same hardware and software
  • Resources can be shared but there should not be performance conflict between resources
  • Multiple Apps will be running in parallel
  • Spike in resource consumption of one app should not affect other application's performance
  • SLA defined for performance need to be met for all hosted apps
Elasticity
  • Applications will have projected capacity vs actual capacity
  • Based on a ball park figure but actual load will be measure when the product is implemented
  • Scale as you need
Scalable
  • Process several requests, Store Huge data, Analytics on top of data are offerings
Other key aspects include Availability, Security, Metering, Global APIs, Load Balancing, Simple API's

More Detailed Architecture is explained in paper link
  • Overview of Open Stack
  • Apache Traffic Server used as caching proxy server
  • Proxy (Route Traffic through intermediate steps)
  • Reverse proxy vs Forward Proxy (Several Variations)
  • Yahoo has 25K Clusters and 40K Servers
  • Mobstor (Storage for large unstructured files)
  • Sherpa (NOSQL solution from Yahoo)

Happy Learning!!!

September 01, 2013

pyCon India 2013 - Day 2 Session Notes

Please find second day session notes
Session #1 – Rasberry PI basics by Sudar Muthu
Good basic session. Speaker presented the content and demo very well. Notes from the session
  • Simplest helloworld program on Rasberry PI is a light blink program
  • Speaker also spoke about controlling devices
  • Using PWM (Pulse width modulation) devices can be managed
  • PWM.py – pull up (Higher Voltage), pull down (Lower Voltage)
  • Protocols – I2C, SPI, Serial. These protocols can be used to talk to devices
  • Interacting with web cam using PyGame
More Reads -  Distributed Computing Tutorials, Author website – HardwareforFun
Session #2 – Robotics Demo
  •  ROS (Robotics Operating System)
  • Author used RasberryPi and arduino Node
Tools
  • Speech Synthesis – Festival and pyFestival
  • Speech Recognition – Gstreamer, Pocket Sphinx
  • Artificial Intelligence – AIML, pyAIML (Artifical Intelligence Markup language)
  • GUI – QT and pyQT
Author site – Technolabz, Lentin Joseph
Session #3 – arduino and Internet of Things
Arduino – Open Source Electronics Prototyping platform
Advantages of arduino
  • Easy to use
  • Cheap
  • Open Hardware
  • Open Documents
Components
  • Hw: device : electronic prototyping board
  • Sw:bootloader
  • Sw:libraries
  • Sw: IDE
  • Interfacing – Connected to computer using Bluetooth / USB
Tools
Use Cases
  • Talk over Serial, RF, Ethernet
  • Attach Sensor and relay other readings
  • Attach Actuators and make things move
  • Connecting devices through web
  • Security Sensor, email on touch
  • Author - Avik Dhupar
  • http://www.arduino.cc/
Session #4 - Testing tools Sessions (Open Source Tools)
  • Fabric for distributed testing (This deployment tool can be used for distributed computing testing  )
  • STAF IBM Test Automation Framework Tool -
  • Nitrate Test case management tools -
  • Test link – Test case management tool
  • Beaker Project – Managing Automated Tests
Session #5 – Web Scraping
  • Author used http://scrapy.org/
  • Pablo Hoffman is the scappy developer
  • Author Anuvrat Parashar provided examples on crawaling web, collecting data, extracting information from collected data 
More Tools

Happy Learning!!!

August 31, 2013

Day1 - pyCon2013 Notes


Hello World!!
After a long time updating my notes, learning’s in this post.
Some good one liner’s from Pycon2013
  • WLT – We Love Typing
  • DRY – Don’t Repeat Yourself
Today I attended Pycon 2013. This was my first python focussed learning session. Sessions were pretty good. Summarizing my notes of today's session
Session #1 - The first session was log management in inmobi 
  • They have a framework built to store, manage all kinds of logs (Web Server Logs, DB Logs, Application logs, Cron Tab Logs
  • Three steps were followed in collecting logs
  • Collection of Logs – Apply Patterns – Fetch Logs
  • Tools used include logstash, grok (ships with parse patterns) - http://logstash.net/ 
Author also provided other alternatives and tools for implementation
  • Transport – flume, rsyslog, conduit, scribe
  • Search and Analytics – hadoop, graylogz, elsa
  • Storage – HDFS, Cassandra, Elastic Search
  • Apache Falcon
Jordan Sissel is the brain behind logstash. His video session is also available in link
Session #2 – Django Framework beyond Basics
It was a well presented session. Although this was my first session on Django I could get the feel of the framework. Presenter Arun (http://Arunrocks.com)
Django – Web Framework. This is built on principles of Rapid App Development. How a typical web request is handled in Django framework 
  • Models – Matter
  • Views – Thinkers
  • Templates – Should be dumb
Good Things about Django Framework
  • Good Admin Interface
  • Security
  • Great Documentation
  • Stable
  • Batteries Included
Basic Definitions of Query Set (Object that interfaces with DB), Media Separate etc. Python implements class based views. Earlier it was function based generic views
Tools Discussed
  • IDE – Pychan, PyDev, Emacs, Vi, Sublimetext
  • Deployment Tools – Chef, Puppet, fabric
  • Security Checks – ponycheckup.com for Django based sites
  • ORM – SQLAlchemy
Session #3 – Third Session was Rapid development & integration of Real Time Communication in Websites
The session as pretty interactive. Demonstrating realtime video chat using google webrtc. https://github.com/cjgiridhar
Demonstrated the setup
  • End User Requests sent to Tornado Web Server
  • Using Web Socket / Ajax communication happens (video chat / live chat)
  • Chat messages saved in Redis database
Tools
  • Webserver – flask, bottle, tornado
  • Javascript and webRTC
Chetan’s website - link
More Links - pythontutor

Happy Learning!!!

July 16, 2013

HSQLDB - Getting Started

This post is on learning HSQLDB. HSQLDB is Rdbms db written in java
  1. Download HSQLDB from link 
  2. Getting started guide useful from link 
  3. Java is already installed on my laptop. Adding JAVA_HOME , PATH and CLASSPATH provided in link
Lets get started and try out some basic examples
Step 1 - To Start and create a DB
  
From command line
 
 Command text -  C:\Program Files\Java\jre7\bin>java.exe -cp "E:\HSQLDB\hsqldb-2.3.0\hsqldb-2.3.0\hsqldb\lib\hsqldb.jar" org.hsqldb.server.Server --database.0 file:mydb --dbname.0 xdb
Step 2 - DB files would be created as below
 
 
Step 3 - Opening the DB Manager
 
 
Command text - C:\Program Files\Java\jre7\bin>java.exe -cp "E:\HSQLDB\hsqldb-2.3.0\hsqldb-2.3.0\hsqldb\lib\hsqldb.jar" org.hsqldb.util.DatabaseManagerSwing
 
Command text - Connecting to DB Instance
jdbc:hsqldb:hsql://localhost/xdb

Step 4 - Basics on Table Creation

The three types of persistent tables are MEMORY tables, CACHED tables and TEXT tables
  • Memory Tables - Data stored in Files
  • Cached Tables - Cached detail remains in memory not in File
  • Text Files - Use CSV Supported Files
Step 5 - Table Creation
 
CREATE TABLE PUBLIC.TEST_TABLE
 (COL1 INTEGER NOT NULL,
 COL2 VARCHAR(25) NOT NULL,
 PRIMARY KEY (COL1))
 
Step 6 - Load Data and Select Query
 
INSERT INTO "PUBLIC"."TEST_TABLE"( "COL1", "COL2" ) VALUES ( 10, 'Test')
INSERT INTO "PUBLIC"."TEST_TABLE"( "COL1", "COL2" ) VALUES ( 20, 'Ram')
INSERT INTO "PUBLIC"."TEST_TABLE"( "COL1", "COL2" ) VALUES ( 30, 'Raj')
INSERT INTO "PUBLIC"."TEST_TABLE"( "COL1", "COL2" ) VALUES ( 40, 'Ravi')
INSERT INTO "PUBLIC"."TEST_TABLE"( "COL1", "COL2" ) VALUES ( 50, 'Raja')
 
SELECT * FROM "PUBLIC"."TEST_TABLE"
 
Step 7 - MVCC Basics
 
Reading only committed data, Driven by isolation level settings. More details in link

Supported Isolation levels
  • SET TRANSACTION READ ONLY
  • SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
  • SET TRANSACTION READ WRITE, ISOLATION LEVEL READ COMMITTED 

Step 8 - MVCC Example

Start Two Instances of Data Manager

Window1 - Run below query

SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
SET DATABASE TRANSACTION CONTROL MVCC;
set autocommit false;
INSERT INTO "PUBLIC"."TEST_TABLE"( "COL1", "COL2" ) VALUES ( 10, 'Test')
INSERT INTO "PUBLIC"."TEST_TABLE"( "COL1", "COL2" ) VALUES ( 20, 'Ram')
INSERT INTO "PUBLIC"."TEST_TABLE"( "COL1", "COL2" ) VALUES ( 30, 'Raj')
INSERT INTO "PUBLIC"."TEST_TABLE"( "COL1", "COL2" ) VALUES ( 40, 'Ravi')
INSERT INTO "PUBLIC"."TEST_TABLE"( "COL1", "COL2" ) VALUES ( 50, 'Raja')
commit
INSERT INTO "PUBLIC"."TEST_TABLE"( "COL1", "COL2" ) VALUES ( 70, '7Ravi')
INSERT INTO "PUBLIC"."TEST_TABLE"( "COL1", "COL2" ) VALUES ( 80, '8Raja')
 
Window2 - Run below query

SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
SELECT * FROM PUBLIC.TEST_TABLE;

The Result from Window2 will not include 70, and 80 the uncommitted records


Step 9 - Explore the system created files - mydb.script, mydb.log file using notepad, You would see details on DB Settings and properties

References

Happy Learning!!!

  

June 26, 2013

TSQL Learning

This post is based on very good reading from Article T-SQL Misconceptions - JOIN ON vs. WHERE. These are new learning's for me


1. For Readability - Refrain from using search arguments in the ON clause, and use the WHERE clause instead
2. When Outer Join is Converted to Inner Join - Adding references to the table in the right side of a JOIN to the WHERE clause will convert the OUTER JOIN to an INNER JOIN. I'm trying out Example Code to reproduce the scenario. This Shows the behavior when Left Join in Query is Converted to Inner Join during Execution.


Step 1 - Create Tables

CREATE TABLE Table1(Col1 INT IDENTITY(1,1) PRIMARY KEY, Col2 VARCHAR(40), Col3 VARCHAR(50))

CREATE TABLE Table2(Col1 INT IDENTITY(1,1) PRIMARY KEY, Col2 VARCHAR(40), Col3 VARCHAR(50))


Step 2 - Populate Data

DECLARE @J INT
 SET @J = 1
 WHILE 1 = 1
 BEGIN
 IF @J < 1000
     INSERT INTO Table1(Col2, Col3)
     VALUES((CONVERT(VARCHAR(20),@J)+'VALUE'), (CONVERT(VARCHAR(20),@J)+'VALUE'))
     INSERT INTO Table2(Col2, Col3)
     VALUES((CONVERT(VARCHAR(20),@J)+'VALUE'), (CONVERT(VARCHAR(20),@J)+'VALUE'))
     SET @J = @J + 1
     IF @J > 1000
     BREAK;
 END


Step 3 - Run Queries

SELECT
 N1.Col1, N2.Col2
 FROM Table1 N1
 LEFT JOIN Table2 N2
 ON N1.Col1 = N2.Col1
 WHERE N2.Col1 = 100
 

SELECT
 N1.Col1, N2.Col2
 FROM Table1 N1
 LEFT JOIN Table2 N2
 ON N1.Col1 = N2.Col1
 WHERE N1.Col1 = 120


Step 4 - Execution Plan


Happy Learning!!!

June 21, 2013

Big Data Updates

Interesting Big Data Updates


[Good Read] - Big Data Analysis Takes a Big Bite Out of HPC Workloads: IDC
Summary - High Performance Servers used to run / benchmark big data workloads

[Learning Resource] - Hadoop 101: Programming MapReduce with Native Libraries

[Good Read] - Big Data Transforming Traditional DW Ecosystem - Why Hadoop and Solr in DataStax Enterprise?

Summary 
  • NOSQL feasible alternative competing OLTP Space
  • Big Data Replacing OLAP Ecosystem
Happy Reading!!!

MYSQL Exercises

Couple of learning exercises while working on MYSQL. Earlier MYSQL post please refer link
Learning #1 - Load data from flat file
Created a Temp table, Loaded bulk data using command - LOAD DATA LOCAL INFILE

CREATE TABLE TestTable2 ( A INT NOT NULL, B INT NOT NULL, C INT NOT NULL, D INT NOT NULL );

Dataset looks as below



LOAD DATA LOCAL INFILE 'E:\\DataUpload50K.txt' INTO TABLE TestTable2 FIELDS TERMINATED BY ','  LINES TERMINATED BY '\n';

Reference - link1 , link2
Learning #2 - Enable Remote Access to MYSQL

Execute the Grant All Privileges command by specifying username, IP Address of remote host and password to logon
GRANT ALL PRIVILEGES ON *.* TO 'username'@'XX.XX.XX.XX' IDENTIFIED BY 'pwd';
Encountered error codes 1130, 1045
Reference - link1, link2, link3

Learning #3 - Show System Configuration Settings
Command - SHOW VARIABLES
Reference - link1

Happy Learning!!!

June 19, 2013

heidisql - SQL Interface to MSSQL and MYSQL

This quick post is based on tool shared by my colleague Ambuj. Heidisql - SQL Editor for MSSQL and MYSQL

Step 1 - Pretty simple and quick to install. Downloaded and Installed it from link

Step 2 - You can setup sessions and connect to MSSQL / MYSQL instances. I have used SSMS, Atlantis SQL Explorer. This single interface for both MYSQL and MSSQL is very useful.




Happy TSQL coding !!!

May 15, 2013

TSQL Formatting Tools

Back to blogging after a month long break. Things were quite busy @ work. This post is to focus on TSQL formatters. There are several code formatters that I had come across recently.

1. SQL Formatter for Notepad++ - Download and install Notepad++ plugin from link. Copying the DLL's and checking the Notepad++ Option listed below

Step 1. Copy the DLL's in plugin directory of Notepad++


Step 2. NotePad++ SQL Formatter Options


2. Online SQL Formatter - Link

3. PoorSQL - Online SQL FormatterLink

4. SSMS Tools pack - SSMS Addons - Link

5. Instant SQL Formatter - Online version - Link

6. Atlantis Interactive SQL Server IDE - Very good IDE for MSSQL - Link

7. SQLInform - SQL Online formatter - Link

Other Free Useful Tools
1. Programmers Notepad
- Link
2. TextPad - Link
3. Notepad++ - Link

Happy Learning!!!

March 17, 2013

Big Data Conference Tech Talks Talks

Please find Tech Talk videos of Interesting Big Data Sessions. Please refer session notes from previous posts

Big Data Analytics @ InMobi



Messaging architecture at Facebook




More Interesting Tech Talk Videos Please check hasgeek TV


Happy Learning!!!

March 09, 2013

SQLite and Java Client


This post is on trying out SQLite and Java Client for querying databases. There were several results returned from google search. The following post Connect Java with SQLite using JDBC Tutorial

Download Links
WIN 32 SQLite - link
sqlite-jdbc.jar - link

Pretty much post was self-explanatory. Posting screenshots of the same project

Step 1 - Running Sqlite ( Pretty easy - console app)


Step 2 - DB and Table Creation


Files Created for Employee



Step 3 - Create a Java Application and add SQLite JAR reference 



Step 4 - Java Project Console Results - Path Changed to Where DB is present



Happy Learning!!!

February 24, 2013

Selenium Builder - Magic Tool for Web Testing

Selenium Builder tool caught my attention from recent selenium post. Selenium IDE is available past few years. It is very useful, robust to get started. The challenging problem for me was fixing XPATH related issues prior to web driver implementation. Selenium Builder seems to remove some of those pain points. In a quick dry run I found below things impressive about Selenium builder.


1. Converting from Selenium 1 to 2
2. Alternatives Suggestions for actions is amazing

3. Run Locally or against Selenium Instance


Earlier we have seen BITE - Browser Integration Test Environment - File Bugs from Browser Level

We Can Simplifying Manual Web Test Process by using
  • Selenium Builder to Records your exploratory tests and replay it for every build
  • Any error encountered File Bugs using BITE
  • Filing bugs is simpler, lot of details attached with bugs filed using BITE and reduces bug logging time
Selenium IDE + Selenium Builder + BITE are powerful set tools for web testers and also right set of tools to get started with automation

Happy Re-Learning!!!

February 19, 2013

Web Test Automation Fundas - One Liners

Below notes are based on my personal learning's from project's I have worked on. This presentation from Bindu - Design Patterns in Automation Testing is the inspiration for this post.

Learning Tip #1 - Page Object - Create Class for Each Page in a website. Ex- HomePage, SearchPage Each Page will have a class on its own. These classes will have methods to simulate the functionality. All example posts on Developing Automation Framework in my blog is based on Page Object pattern

Learning Tip #2 - Factory Pattern - You will still have a single class for each page in your website. When you support multiple locales Factory Pattern is useful to simulate the method based on the locale specified in Test Run. Please refer this earlier post on Data Driven Testing and Design patterns. Example for Factory Pattern is provided in the example

Learning Tip #3 - Data Driven Testing - Running the same test for different Data Sets. Basically Automation Framework should be configurable to run against dynamic data sets (not limited to fixed data sets). Although this is a continuous challenge but this is achievable

Learning Tip #4 - Validation Procedures - Validating data need to be performed using independent Service Calls / DB Queries / Prepared Expected Results output. Use of XML to read configurable parameters, storing DB queries of different DB platforms (DB2, MSSQL, MYSQL..) would help to dynamically fetch queries based on the platform tests are executed


More Reads http://code.google.com/p/selenium/wiki/DesignPatterns
Page Object Pattern and how to use Selenium

Happy Re-Learning!!!

February 12, 2013

Database Internals [Good Reads]

Please download and read white paper 'OLTP Through the Looking Glass, and What We Found There'

The whitepaper provides more details on current architecture of OLTP RDBMS Systems. I can closely relate it with MSSQL Server. It covers the key architecture details. The Paper evaluates options by eliminating some of the features based on current hardware systems and demonstrates significant performance gains
 
At a beginner level some of architecture keywords / concepts good to learn are
  • WAL (Write Ahead Logging)
  • Log Manager
  • Buffer Manager
  • Two Phase Locking
  • Concurrency 
  • Eventual Consistency
  • Transaction Manager
I am posting some relavant simple explanations of the same based on MSDN documentation. WAL -
 
Write Ahead Logging (Source - Link)
 
 
Buffer Manager - Page is fetched to main memory through Buffer Manager (Source - Link)
  

Two Phase Locking - A detailed note of how transaction is executed from Brent Ozar Post
 
Concurrency Control - Governed by Isolation levels, Earlier discussed in same blog
 
Log Manager - Manages Logging, WAL
 
Eventual Consistency - Over a period of time the data will be consitent across different nodes (databases) distributed over network. Read More @ Consistency models in nonrelational dbs 
 
 
More Reads  
 
Happy Reading!!! 

January 26, 2013

Good Tech Talk - Making Better Software - QA & DEV

Today I checked this tech talk. This is very very good. **MUST VIEW** for passionate DEV / QA / IT Pro. Complete Tech Talk is inspiring, offers a lot of improvement ideas
 

Between 24:31 to 30:52 minutes

Very Good Joel talk on test skills, testing competency and also covers **Agile QA**
 
Summary of QA Talk
  • Benefits of Having QA in a Team
  • Benefits of Rapid Feedback between Dev & QA
  • Tools for testing + Functional testing both are critical
Between 32:34 to 34:37 minutes
  • How Writing code vs Developing a Great product differs?
Between 44:18 to 48:54
  • Intrinsic and Extrinsic motivation
  • Intrinsic Motivation matters!!
 
Hope you enjoyed the tech talk. Happy Listening!!!

January 06, 2013

Good GTAC Testing Talks


GTAC 2011: The Latest in Google Test Tools

Tools covered in the session are BITE and Quality Bots

BITE - Browser Integrated Test Environment
  • Record Browser Actions, File Bugs Automatically without switching from your browser, Replay Recorded Actions
  • BITE is available as a Chrome Extension 
  • Records User Actions @ Browser Level
  • Track Actions while finding UI bugs during Exploratory Testing  
  • Bug Filing from Browser itself
  • Include Screenshots, Repro Steps while filing bug
Quality Bots - Automated Layout Issues Detection in Chrome browsers

With the Tools Implemented QA Cycle works as below
  • Run Quality bots to detect layout issues
  • Allow exploratory testing for Manual testers and capture test steps, screen shots, related data using BITE chrome extensions for bugs identified
  • Automated Bug Filing using BITE from browser itself
Tools + Exploratory Testing with test steps captured makes it easier to reproduce bugs

More Interesting Reads

Happy Learning!!!