<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd"
xmlns:rawvoice="http://www.rawvoice.com/rawvoiceRssModule/"
>

<channel>
	<title>Solution Hacker &#187; etl</title>
	<atom:link href="http://www.solutionhacker.com/tag/etl/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.solutionhacker.com</link>
	<description>This blog provides solutions for enterpreneurs!</description>
	<lastBuildDate>Sun, 05 Feb 2012 00:45:27 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=109</generator>
<!-- podcast_generator="Blubrry PowerPress/2.0.4" -->
	<itunes:summary>This blog provides solutions for enterpreneurs!</itunes:summary>
	<itunes:author>Solution Hacker</itunes:author>
	<itunes:explicit>no</itunes:explicit>
	<itunes:image href="http://www.solutionhacker.com/wp-content/plugins/powerpress/itunes_default.jpg" />
	<itunes:subtitle>This blog provides solutions for enterpreneurs!</itunes:subtitle>
	<image>
		<title>Solution Hacker &#187; etl</title>
		<url>http://www.solutionhacker.com/wp-content/plugins/powerpress/rss_default.jpg</url>
		<link>http://www.solutionhacker.com</link>
	</image>
		<item>
		<title>Hibernate vs iBatis</title>
		<link>http://www.solutionhacker.com/implement-your-idea/build-your-website/hibernate-vs-ibatis/</link>
		<comments>http://www.solutionhacker.com/implement-your-idea/build-your-website/hibernate-vs-ibatis/#comments</comments>
		<pubDate>Thu, 26 Feb 2009 18:58:41 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[Design]]></category>
		<category><![CDATA[Site Building]]></category>
		<category><![CDATA[data warehouse]]></category>
		<category><![CDATA[etl]]></category>
		<category><![CDATA[hibernate]]></category>
		<category><![CDATA[iBatis]]></category>
		<category><![CDATA[reporting]]></category>

		<guid isPermaLink="false">http://www.solutionhacker.com/?p=216</guid>
		<description><![CDATA[<p><strong>Hibernate </strong>is great. However, I don't see it fits all the data access requirements. At its core, it is an <strong>ORM </strong>tool that helps you to map your <strong>object model</strong> to <strong>relational model</strong>. If you have full control of your relational model and perform lots of <strong>CRUD</strong> operations, it is certainly a great tool for you. Its transparent persistence, 2 level caching, dirty checking, lazy/ eager data retrieval and sql generation indeed can save us lots of development time. However, one tool doesn't fit all !! Why not?</p>
<p>In my current company, I have created a reporting tool that interfaces with dimensional model in data warehouse. In this setting, you will deal with <strong>star schema</strong> with <strong>denormalized dimensions</strong>.&#160; Often time, I need to tune the query performance via looking into explain plan. Without full control of SQL, my job will be hard to achieve. Apart from that, reporting tool often issues read-only set based queries to the data warehouse. The resultset returned doesn't fit into my OO model at all. Again, Hibernate just doesn't fit in. People in my company argue that I should use <strong>named query</strong> in Hibernate for the sake of sticking with the standard. I am like ok, whatever... I have known a tool called <strong>iBatis </strong>that I can achieve my job cleanly. Why the hell I would have motivation to try named query that basically a way to by-pass the ORM model to query database. What benefit I will get from this? The cache? We are using ETL to update our fact and dimension in the data warehouse, not by my reporting app. Unless the ETL throws me an event when the update cycle is finished so I can flush my cache, I simply don't think it gives me lots of help.</p>
<p>Anyway, it is just my own little perspective. You don't have to agree with me. The point here is not that I don't like Hibernate but I don't like to be pushed to use it only because it is "standard" to someone. If Hibernate could help me to construct my sql based on user input, stream my result directly to my presentation layer, populate my model automatically based on mapping I provided, detect data warehouse changes and take care my cache, then I am more happy to adopt it in my reporting app. Otherwise, I would not be eager to dump my <strong>iBatis </strong>DAO layer unless I get no choice under the political game.</p>
<h2>Reference</h2>
<ol>
    <li><a href="http://www.nofluffjuststuff.com/media.jsp?id=19">http://www.nofluffjuststuff.com/media.jsp?id=19</a></li>
    <li><a href="http://www.javalobby.org/articles/hibernate-query-101/">http://www.javalobby.org/articles/hibernate-query-101/</a></li>
    <li><a href="http://www.javabeat.net/tips/108-how-to-use-named-parameters-and-named-query-i.html">How to use named parameters and named query in Hibernate?</a></li>
    <li><a href="http://www.ibm.com/developerworks/java/library/j-genericdao.html">Don't repeat your DAO</a>&#160;</li>
</ol>]]></description>
			<content:encoded><![CDATA[<p><strong>Hibernate </strong>is great. However, I don&#8217;t see it fits all the data access requirements. At its core, it is an <strong>ORM </strong>tool that helps you to map your <strong>object model</strong> to <strong>relational model</strong>. If you have full control of your relational model and perform lots of <strong>CRUD</strong> operations, it is certainly a great tool for you. Its transparent persistence, 2 level caching, dirty checking, lazy/ eager data retrieval and sql generation indeed can save us lots of development time. However, one tool doesn&#8217;t fit all !! Why not?</p>
<p>In my current company, I have created a reporting tool that interfaces with dimensional model in data warehouse. In this setting, you will deal with <strong>star schema</strong> with <strong>denormalized dimensions</strong>.&#160; Often time, I need to tune the query performance via looking into explain plan. Without full control of SQL, my job will be hard to achieve. Apart from that, reporting tool often issues read-only set based queries to the data warehouse. The resultset returned doesn&#8217;t fit into my OO model at all. Again, Hibernate just doesn&#8217;t fit in. People in my company argue that I should use <strong>named query</strong> in Hibernate for the sake of sticking with the standard. I am like ok, whatever&#8230; I have known a tool called <strong>iBatis </strong>that I can achieve my job cleanly. Why the hell I would have motivation to try named query that basically a way to by-pass the ORM model to query database. What benefit I will get from this? The cache? We are using ETL to update our fact and dimension in the data warehouse, not by my reporting app. Unless the ETL throws me an event when the update cycle is finished so I can flush my cache, I simply don&#8217;t think it gives me lots of help.</p>
<p>Anyway, it is just my own little perspective. You don&#8217;t have to agree with me. The point here is not that I don&#8217;t like Hibernate but I don&#8217;t like to be pushed to use it only because it is &#8220;standard&#8221; to someone. If Hibernate could help me to construct my sql based on user input, stream my result directly to my presentation layer, populate my model automatically based on mapping I provided, detect data warehouse changes and take care my cache, then I am more happy to adopt it in my reporting app. Otherwise, I would not be eager to dump my <strong>iBatis </strong>DAO layer unless I get no choice under the political game.</p>
<h2>Reference</h2>
<ol>
<li><a href="http://www.nofluffjuststuff.com/media.jsp?id=19">http://www.nofluffjuststuff.com/media.jsp?id=19</a></li>
<li><a href="http://www.javalobby.org/articles/hibernate-query-101/">http://www.javalobby.org/articles/hibernate-query-101/</a></li>
<li><a href="http://www.javabeat.net/tips/108-how-to-use-named-parameters-and-named-query-i.html">How to use named parameters and named query in Hibernate?</a></li>
<li><a href="http://www.ibm.com/developerworks/java/library/j-genericdao.html">Don&#8217;t repeat your DAO</a>&#160;</li>
</ol>
]]></content:encoded>
			<wfw:commentRss>http://www.solutionhacker.com/implement-your-idea/build-your-website/hibernate-vs-ibatis/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Common DBA jobs</title>
		<link>http://www.solutionhacker.com/data-intelligence/data-store/common-dba-jobs/</link>
		<comments>http://www.solutionhacker.com/data-intelligence/data-store/common-dba-jobs/#comments</comments>
		<pubDate>Mon, 17 Nov 2008 00:46:06 +0000</pubDate>
		<dc:creator>admin</dc:creator>
				<category><![CDATA[Data Store]]></category>
		<category><![CDATA[etl]]></category>
		<category><![CDATA[export]]></category>
		<category><![CDATA[mysql]]></category>

		<guid isPermaLink="false">http://www.solutionhacker.com/?p=192</guid>
		<description><![CDATA[<p><u><strong>Export schema/ data out from mysql</strong></u></p>
<p>To export schema and/or data, you can use mysqldump command:</p>
<blockquote>
<p>mysqldump -u [username] -p[password] -d&#160;[schema_name] &#62; [filename].sql</p>
</blockquote>
<ol>
    <li>-d means no data (just gives me the schema).</li>
    <li>-B is needed for multiple schema output</li>
    <li>-h (hostname)</li>
</ol>
<p><u><strong>Export data out from postgresql</strong></u></p>
<ol>
    <li><a href="http://www.mkyong.com/database/how-to-export-table-data-to-file-csv-postgresql/">Export table data from postgresql to csv format</a></li>
    <li><a href="http://www.mkyong.com/database/backup-restore-database-in-postgresql-pg_dumppg_restore/">Backup and restore database in postgresql</a></li>
</ol>
<p>However, if you want to export sql result set to csv in postgresql, you can consider to use COPY functionality.</p>
<blockquote>
<p>COPY (&#160;select statement&#160;) TO STDOUT WITH CSV<br />
<span id="intelliTxt" name="intelliTxt">COPY stock FROM 'mydir/Stock.csv';</span></p>
</blockquote>
<p><u><strong>Run sql script using mysql command</strong></u></p>
<p>To run the scripts as input, we can use the following command:</p>
<blockquote>
<p>mysql [schema_name] -u [username] -p[password] &#60; [filename].sql</p>
</blockquote>
<p><u><strong>SQL Tips</strong></u></p>
<p>There are times we want to put logic in SQL but not writing store procedure. Here are some of using functions that may get you there:</p>
<ul>
    <li>Conditional statement - CASE WHEN xxx THEN abc WHEN yyy THEN bbc ...ELSE ccc END</li>
</ul>
<blockquote>
<p>UPDATE Account SET Sales_Location__c =<br />
&#160;&#160;&#160; CASE WHEN Sales_Country__c != '' THEN Sales_Country__c WHEN Country__c != '' THEN Country__c<br />
ELSE '--' END</p>
</blockquote>
<ul>
    <li>COALESCE (input1, input2,....) - This function takes in as many parameter as you want and return you the first non-NULL parameter. Suppose we have a table A having 3 columns FullName,&#160;CompleteName and DisplayName. Any of these columns can contain null values. Now we want to select the DisplayName from this table, but if it is null, then return FullName, if that is also null then return CompleteName. We can easily perform the same in one select statement as: (<a href="http://databases.aspfaq.com/database/coalesce-vs-isnull-sql.html">COALESCE vs ISNULL</a>)</li>
</ul>
<blockquote>
<p>SELECT COALESCE(DisplayName, FullName, CompleteName) From A</p>
</blockquote>
<p><u><strong>&#160;ETL</strong></u></p>
<p>In mysql, you can export a table from db1 and import it to db2 remotely. For example, in db2 host, you can issue the following command:</p>
<blockquote>
<p>/usr/bin/mysqldump - -force - -compress - -opt -u [username] -p[password] -h [hostname] db1 [table] &#124; /usr/bin/mysql -u [username] -p[password] -D db2</p>
</blockquote>]]></description>
			<content:encoded><![CDATA[<p><u><strong>Export schema/ data out from mysql</strong></u></p>
<p>To export schema and/or data, you can use mysqldump command:</p>
<blockquote>
<p>mysqldump -u [username] -p[password] -d&#160;[schema_name] &gt; [filename].sql</p>
</blockquote>
<ol>
<li>-d means no data (just gives me the schema).</li>
<li>-B is needed for multiple schema output</li>
<li>-h (hostname)</li>
</ol>
<p><u><strong>Export data out from postgresql</strong></u></p>
<ol>
<li><a href="http://www.mkyong.com/database/how-to-export-table-data-to-file-csv-postgresql/">Export table data from postgresql to csv format</a></li>
<li><a href="http://www.mkyong.com/database/backup-restore-database-in-postgresql-pg_dumppg_restore/">Backup and restore database in postgresql</a></li>
</ol>
<p>However, if you want to export sql result set to csv in postgresql, you can consider to use COPY functionality.</p>
<blockquote>
<p>COPY (&#160;select statement&#160;) TO STDOUT WITH CSV<br />
<span id="intelliTxt" name="intelliTxt">COPY stock FROM &#8216;mydir/Stock.csv&#8217;;</span></p>
</blockquote>
<p><u><strong>Run sql script using mysql command</strong></u></p>
<p>To run the scripts as input, we can use the following command:</p>
<blockquote>
<p>mysql [schema_name] -u [username] -p[password] &lt; [filename].sql</p>
</blockquote>
<p><u><strong>SQL Tips</strong></u></p>
<p>There are times we want to put logic in SQL but not writing store procedure. Here are some of using functions that may get you there:</p>
<ul>
<li>Conditional statement &#8211; CASE WHEN xxx THEN abc WHEN yyy THEN bbc &#8230;ELSE ccc END</li>
</ul>
<blockquote>
<p>UPDATE Account SET Sales_Location__c =<br />
&#160;&#160;&#160; CASE WHEN Sales_Country__c != &#8221; THEN Sales_Country__c WHEN Country__c != &#8221; THEN Country__c<br />
ELSE &#8216;&#8211;&#8217; END</p>
</blockquote>
<ul>
<li>COALESCE (input1, input2,&#8230;.) &#8211; This function takes in as many parameter as you want and return you the first non-NULL parameter. Suppose we have a table A having 3 columns FullName,&#160;CompleteName and DisplayName. Any of these columns can contain null values. Now we want to select the DisplayName from this table, but if it is null, then return FullName, if that is also null then return CompleteName. We can easily perform the same in one select statement as: (<a href="http://databases.aspfaq.com/database/coalesce-vs-isnull-sql.html">COALESCE vs ISNULL</a>)</li>
</ul>
<blockquote>
<p>SELECT COALESCE(DisplayName, FullName, CompleteName) From A</p>
</blockquote>
<p><u><strong>&#160;ETL</strong></u></p>
<p>In mysql, you can export a table from db1 and import it to db2 remotely. For example, in db2 host, you can issue the following command:</p>
<blockquote>
<p>/usr/bin/mysqldump &#8211; -force &#8211; -compress &#8211; -opt -u [username] -p[password] -h [hostname] db1
<div class="su-table su-table-style-1"></div>
<p> | /usr/bin/mysql -u [username] -p[password] -D db2</p>
</blockquote>
]]></content:encoded>
			<wfw:commentRss>http://www.solutionhacker.com/data-intelligence/data-store/common-dba-jobs/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

