iDatamining.org

I am looking for projects to work on
Please contact with me at yiyu.jia@BostonInfoPro.com!

Thursday, July 24, 2014

Asgard 1.5 Config.groovy file format

I tried to setup Asgard 1.5 today and find its initializing process has been changed a little bit. I remember I will be asked for aws credential information when I launched asGard first time. But, for Asgard 1.5 (ASGard 1.4.2 has same situation), I got error message like "com.amazonaws.AmazonClientException: Unable to load AWS credentials from any provider in the chain". After searching online, I make it work by setting up environment variable AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

In fact, we can just directly create the asgard configuration file ~/.asgard/Config.groovy. The format of the file is as below.


grails {
        awsAccounts=['YourAccountIDasDigitalNumber']
        awsAccountNames=['YourAccountIDasDigitalNumber':'YourAccountName']
}
secret {
        accessId='YourAccessID'
        secretKey='YourAccessSecretKey'
}
cloud {
        accountName='YourAccountName'
        publicResourceAccounts=['amazon']
}

Hope above tip will save others time. In fact, I solve the problem after reading this discussion on google asgard group.

Friday, July 11, 2014

can we leave missing data as null when load source data into data warehouse?

No! 


because

NULL != NULL

Think about when you do a outer join and some NULL was created after outer join. Then, SQL programmers, who will create SQL query on aggregate tables, will not be able to identify the source of NULL.