Amazon Simple Notification Service Command Lines


Hits: 420  

Install the AWS CLI Using the Bundled Installer

Download the AWS CLI Bundled Installer.

# curl “https://s3.amazonaws.com/aws-cli/awscli-bundle.zip” -o “awscli-bundle.zip”
# unzip awscli-bundle.zip
# chmod a+x ./awscli-bundle/install
# ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws

Configure AWS CLI

# aws configure
AWS Access Key ID [None]: XXXXXXXXXXXXXXXXXXXXX
AWS Secret Access Key [None]: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Default region name [None]: us-east-1
Default output format [None]: json

# cat ~/.aws/credentials

[default]
aws_access_key_id=XXXXXXXXXXXXXXXXXXXXX
aws_secret_access_key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

# cat ~/.aws/config

[default]
region=us-east-1
output=json

AWS SNS Common actions

– Create a communication channel to send messages and subscribe to notifications
– Create a platform application for mobile devices
– Subscribe an endpoint to a topic to receive messages published to that topic
– Publish a message to a topic or as a direct publish to a platform endpoint
– Publish a text message (SMS) to a phone number

Create AWS SNS Topic

# aws sns create-topic –name My-Blog
{
“TopicArn”: “arn:aws:sns:us-east-1:xxxxxxxxxxxx:My-Blog”
}

Now you can subscribe to Topic using these protocols

http, https, email, email-json, sms, sqs, application, lambda

check more here: http://docs.aws.amazon.com/cli/latest/reference/sns/subscribe.html

Subscribe to AWS SNS Topic by email

# aws sns subscribe –topic-arn arn:aws:sns:us-east-1:xxxxxxxxxxxx:My-Blog –protocol email –notification-endpoint email@domain.com
{
“SubscriptionArn”: “pending confirmation”
}

AWS SNS Confirm

After Subscription Confirmed it provide SubscriptionArn like arn:aws:sns:us-east-1:xxxxxxxxxxxx:My-Blog:4e2a8654-6b5d-43fe-bf14-2237c8c849f7

AWS SNS Pending Confirm

Subscribe to AWS SNS Topic by Mobile (SMS)

# aws sns subscribe –topic-arn arn:aws:sns:us-east-1:xxxxxxxxxxxx:My-Blog –protocol sms –notification-endpoint 919912345678
{
“SubscriptionArn”: “arn:aws:sns:us-east-1:xxxxxxxxxxxx:My-Blog:a0bb38ca-ece1-4adb-897b-5a21010889e9”
}

Publish message to Topic

# aws sns publish –topic-arn arn:aws:sns:us-east-1:xxxxxxxxxxxx:My-Blog –message “Welcome to My Blog!”
{
“MessageId”: “9f280eb4-b561-5d34-b8c3-5f3be74515b2”
}

AWS SNS Push Message

AWS SNS SMS

Unsubscribe to AWS SNS Topic

# aws sns unsubscribe –subscription-arn “arn:aws:sns:us-east-1:xxxxxxxxxxxx:My-Blog:4e2a8654-6b5d-43fe-bf14-2237c8c849f7”

You can find Available Commands for AWS SNS here
http://docs.aws.amazon.com/cli/latest/reference/sns/index.html#cli-aws-sns

Delete AWS SNS Topic

# aws sns delete-topic –topic-arn “arn:aws:sns:us-east-1:xxxxxxxxxxxx:My-Blog”

Uninstalling AWS CLI

# rm -rf /usr/local/aws
# rm /usr/local/bin/aws

Install the AWS Command Line Interface on Microsoft Windows
http://docs.aws.amazon.com/cli/latest/userguide/awscli-install-windows.html#install-msi-on-windows

Amazon DynamoDB PHP CRUD using aws.phar


Hits: 537  

DynamoDB is fully managed, fast and low cost NoSQL database service by Amazon. DynamoDB allows you to offload the administrative operations and scale highly distributed database cluster.

The AWS SDK PHP enables you to use Amazon Web Services like Amazon S3, EC2, RDS, CloudFront, Redshift, DynamoDB, Push Notification Service (SNS) and more from your PHP code project.

AWS.PHAR Now Download aws-php-sdk Packaged Phar from http://pear.amazonwebservices.com/get/aws.phar

# wget http://pear.amazonwebservices.com/get/aws.phar

You can also download the packaged phar here http://docs.aws.amazon.com/aws-sdk-php/v3/download/aws.phar

Include the SDK in your PHP project require ‘/path/to/aws.phar’;

Now download simple dynamodb-php-wrapper from here https://github.com/masayuki0812/dynamodb-php-wrapper/blob/master/DynamoDBWrapper.php

<?php

require_once 'aws.phar';
require_once 'DynamoDBWrapper.php'; 

$ddb = new DynamoDBWrapper(array(
    'key'    => 'XXXXXXXXXXXXXXXXXXXXX',
    'secret' => 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
    'region' => 'us-east-1'
));

createTable

$result = $ddb->createTable(‘Session’, ‘UserId::N’);
// Create a table ‘Session’, which has a hash key ‘UserId’ as NUMBER.

deleteTable

$result = $ddb->deleteTable(‘Session’);

putItem

$item = array(
‘UserId’ => 1,
‘Name’ => ‘Sandeep’,
);
$result = $ddb->put(‘Session’, $item);

updateItem

$key = array(
‘UserId’ => 1,
);
$update = array(
‘Name’ => array(‘PUT’, ‘SVNLabs’)
);
$result = $ddb->update(‘Session’, $key, $update);

deleteItem

$key = array(
‘UserId’ => 1,
);
$result = $ddb->delete(‘Session’, $key);

Query

$keyConditions = array(
‘UserId’ => 1,
);
$result = $ddb->query(‘Session’, $keyConditions);

Scan

$filter = null;

$result = $ddb->scan(‘Session’, $filter, 1000);

Scan with between date filter

$filter = array(
‘userType’ => “member”,
“date” => array(“BETWEEN”, array(“2016/08/01″,”2016/08/17”))
);
$result = $ddb->scan(“Session”, $filter, 1000);

Other Links

https://docs.aws.amazon.com/aws-sdk-php/v3/guide/getting-started/installation.html
https://github.com/masayuki0812/dynamodb-php-wrapper/wiki
https://github.com/svnlabs/aws-dynamodb-php

Install AWStats with GeoIP Plugin


Hits: 7517  

AWStats is a free analytic tool that generates web, streaming, ftp or mail server statistics, graphically. There are many log analyzer but AWStats is open source Web analytics reporting tool, It can analyze log files from Apache, WebStar, IIS.

Installing AWStats on ubuntu 12.04

# apt-get install awstats

Configuring for yourdomain.com replace yourdomain.com with your real domain

# cp /etc/awstats/awstats.conf /etc/awstats/awstats.yourdomain.com.conf
# vi /etc/awstats/awstats.yourdomain.com.conf

Check you have below settings in /etc/awstats/awstats.yourdomain.com.conf

LogFile=”/var/log/apache2/access.log”
SiteDomain=”yourdomain.com”
HostAliases=”localhost 127.0.0.1 yourdomain.com”
DNSLookup=1
LogFormat=1
AllowFullYearView=3

Generate stats for AWStats for /var/log/apache2/access.log file (Make sure you have fresh log file)

# /usr/lib/cgi-bin/awstats.pl -config=yourdomain.com -update

configuration Apache2 with password

# htpasswd -c /etc/awstats/awstats.pwd admin
# chmod 755 /etc/awstats/awstats.pwd

# vi /etc/apache2/sites-available/awstats

<VirtualHost *:80>
ServerName awstats.yourdomain.com
ServerAdmin admin@yourdomain.com
DocumentRoot /var/www
ErrorLog ${APACHE_LOG_DIR}/error_awstats.log
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/access_awstats.log combined
<Directory “/usr/lib/cgi-bin/”>
<Files “awstats.pl”>
Options +ExecCGI
</Files>
</Directory>
Alias /awstats-icon/ “/usr/share/awstats/icon/”
<Directory /usr/share/awstats/icon>
Options None
AllowOverride All
Order allow,deny
Allow from all
</Directory>
RewriteEngine on
RewriteRule /awstats-icon – [L]
RewriteCond %{QUERY_STRING} !^(.*&)?framename=index(&.*)?$
RewriteCond %{QUERY_STRING} ^(.*&)?framename=.*(&.*)?$
RewriteRule /(.*) /usr/lib/cgi-bin/awstats.pl [L]
RewriteCond %{QUERY_STRING} ^(.*&)?config=([^&]*)(&(.*))?$
RewriteRule /.* /%2?%1%4 [R,L]
RewriteRule /awstats.pl$ / [R,L]
RewriteRule /([^/]*)(/.*)? /usr/lib/cgi-bin/awstats.pl?config=$1 [QSA,L]
<Location />
SetHandler cgi-script
Satisfy any
Order deny,allow
Deny from all
AuthType Basic
AuthName “AWStats Access”
AuthUserFile /etc/awstats/awstats.pwd
require valid-user
</Location>
<LocationMatch /(?!awstats.pl|robots.txt)$>
Require user admin
</LocationMatch>
<Location /awstats-icon>
SetHandler None
Allow from all
</Location>
</VirtualHost>

Enable Virtual Host

# a2ensite awstats
# apache2ctl configtest
# /etc/init.d/apache2 reload

Auto update stats

# crontab -e

*/10 * * * * /usr/share/awstats/tools/update.sh
10 03 * * * /usr/share/awstats/tools/buildstatic.sh
*/10 * * * * /usr/lib/cgi-bin/awstats.pl -config=yourdomain.com -update > /dev/null

Set proper permission on logs

# sudo chmod 644 -R /var/log/apache2/*
# chmod 644 /var/log/apache2

# vi /etc/logrotate.d/apache2   (Replace create 640 root adm with create 644 root adm)

Install GeoIP for AWStats

# apt-get install build-essential zlib1g-dev apg libgeo-ipfree-perl libnet-ip-perl libnet-dns-perl liburi-perl apache2-utils

# cd /usr/src/
# wget http://zlib.net/zlib-1.2.8.tar.gz
# tar xvzf zlib-1.2.8.tar.gz
# cd zlib-1.2.8
# ./configure –prefix=/usr/local/zlib && make && make install

# wget http://www.maxmind.com/download/geoip/api/c/GeoIP-latest.tar.gz
# tar xzvf GeoIP-latest.tar.gz
# cd GeoIP-1.6.0
# ./configure && make && make install

# cpan
> install YAML
> install Geo::IP Geo::IPfree Geo::IP::PurePerl URI::Escape Net::IP Net::DNS Net::XWhois Time::HiRes Time::Local
> quit

# cd /usr/src/
# wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
# gzip -d GeoLiteCity.dat.gz
# cp GeoLiteCity.dat /usr/share/GeoIP/GeoLiteCity.dat

Enable AWSTAT configuation for LoadPlugin

# vi /etc/awstats/awstats.yourdomain.com.conf

Remove comment from /etc/awstats/awstats.yourdomain.com.conf

LoadPlugin=”geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat”
LoadPlugin=”geoip_city_maxmind GEOIP_STANDARD /usr/share/GeoIP/GeoLiteCity.dat”

# /etc/init.d/apache2 reload

AWStats will run here

http://awstats.yourdomain.com/yourdomain.com

AWStats-Stats

AWStats-country

AWStats-hours

Error opening /usr/share/GeoIP/GeoIPCity.dat at /usr/local/share/perl/5.14.2/Geo/IP/PurePerl.pm line [file name in /usr/share/GeoIP]

Make sure you have all files in folder /usr/share/GeoIP/GeoLiteCity.dat else copy

# cp /usr/share/GeoIP/GeoLiteCity.dat /usr/share/GeoIP/GeoIPCity.dat

AWStats-UA

Other log file analysis and reporting tools

http://sawmill.net/
http://goaccess.io/
http://www.apacheviewer.com/
http://www.loganalyzer.net/
http://www.webalizer.org/
http://piwik.org/log-analytics/
http://www.w3perl.com/
http://http-analyze.org/index.php

Create Website Thumbnails using PhantomJS


Hits: 9350  

Hope you read my old article “Create WebThumb using LAMP”, it used for creating web thumbnails from Xvfb virtual framebuffer that is more time and resource consuming.

Shell Script with Xvfb try to open web browsers in real remote server to capture web-thumbs…

That whole setup might take a full day and image quality is very bad….

Thanks to PhantomJS

PhantomJS is a headless WebKit with JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.

Simple Javascript example

console.log(‘Loading a web page’);
var page = require(‘webpage’).create();
var url = ‘http://www.svnlabs.com/’;
page.open(url, function (status) {
//Page is loaded!
phantom.exit();
});

Features:

– Headless Website Testing
– Screen Capture
– Page Automation
– Network Monitoring

Fork on Github https://github.com/ariya/phantomjs

PhantomJS is using WebKit for a real layout and rendering engine, it can be used to capture web page as screenshot or PDF files. PhantomJS can render anything on the web page, it can be used to convert contents in HTML/CSS, SVG and Canvas.

URL2PNG-URL2PDF-HTML2JPG-URL2IMAGE
URL2PNG-URL2PDF-HTML2JPG-URL2IMAGE
// Thumbnail of Webpage using webthumb.js

var page = require('webpage').create(),
    system = require('system'),
    weblink,
    fname;

if (system.args.length !== 3) {
    console.log('Usage: webthumb.js webURL filename');
    console.log('Usage: webthumb.js http://www.svnlabs.com svnlabs.png');
    phantom.exit(1);
} else {
    weblink = system.args[1];
    fname = system.args[2];    
    page.viewportSize = { width: 320, height: 480 };
    page.open(weblink, function (status) {
    if (status !== 'success') {
        console.log('Unable to open webpage!');
    } else {
        page.render(fname);
    }
    window.setTimeout(function () {
      phantom.exit();
    }, 3000);
    });
}

Download webthumb.js

# phantomjs webthumb.js http://www.svnlabs.com svnlabs.png
# phantomjs webthumb.js http://www.svnlabs.com svnlabs.pdf

You can even capture webpage as good quality PDF file 😉

Output:

svnlabs.png
svnlabs.pdf

You can also use examples/rasterize.js for same sort of functions 😉

# phantomjs examples/rasterize.js http://www.svnlabs.com/ svnlabs.png
# phantomjs examples/rasterize.js ‘http://www.svnlabs.com/’ svnlabs.pdf

rasterize.js – rasterizes a web page to image or PDF
render_multi_url.js – renders multiple web pages to images
technews.js – captures Google News as a PNG image

Upload Large Files from Amazon S3 Bucket to Dropbox Folder


Hits: 7022  

Dropbox Uploader is a BASH script (only needs cURL) which can be used to upload, download, list or delete files from Dropbox, an online file sharing, synchronization and backup service.

Usage: ./dropbox_uploader.sh COMMAND [PARAMETERS]…

Amazon S3 to Dropbox
Amazon S3 to Dropbox

Secure:It’s not required to provide your username/password to this script, because it uses the official Dropbox API for authentication process.

START DROPBOX UPLOADER

# chmod +x dropbox_uploader.sh
# ./dropbox_uploader.sh

It will ask “App key”, “App secret” and “Access level (App folder or Full Dropbox)”

As shell script authorize token…

Please visit this URL from your Browser, and allow Dropbox Uploader
to access your DropBox account:

–> https://www2.dropbox.com/1/oauth/authorize?oauth_token=xxxxxxxxxxxxxxx

Dropbox token for Command Line
Dropbox token for Command Line

If everything go well …

> Access Token request… OK

Setup completed!

Now you can download Amazon S3 Bucket files on server using “wget” then you can run command like…

# ./dropbox_uploader.sh COMMAND [PARAMETERS]…
# ./dropbox_uploader.sh upload /var/www/aws/s3/svnlabs.mp4 /DROPBOX-FOLDER/svnlabs.mp4

I have uploaded media file upto 500MB on dropbox successfully using this shell script 😉

Alternate Options:

You can try Node.js for reading files from Amazon S3 and then upload to Dropbox

Useful JavaScript Tools

http://loose-bits.com/2011/10/16/node-sunny-cloud-library.html
https://github.com/nuxusr/Node.js—Amazon-S3
https://github.com/appsattic/connect-stream-s3

https://github.com/evnm/dropbox-node
https://github.com/sintaxi/node-dbox
https://github.com/dropbox/dropbox-js

Amazon S3 Cloud HTML5 MP3 Player


Hits: 4655  

Amazon S3 Cloud HTML5 Player plugin that allows you to link to your amazon S3 bucket and sets up a HTML5 Player playlist (horizontal or vertical). This plugin will support MP3 files only.

Amazon S3 Cloud HTML5 MP3 Player is not associated in any way to amazon.com

Get Self Hosted Amazon S3 Cloud HTML5 MP3 Player for WordPress just in $24

You will need an amazon account to use this.

AWS access info

Add your amazon information amazon key and amazon secret key. Then just add the bucket to your account add folders.

Amazon S3 Cloud HTML5 MP3 Player Options
Amazon S3 Cloud HTML5 MP3 Player Options

Make sure you have public permission for all your music in your S3 buckets and folders.

S3 Management Console
S3 Management Console

Shortcode for Page or Post

[html5aws3mp3 player=”horizontal” bucket=”BUCKET” endpoint=”BUCKET-ENDPINT” folder=”FOLDER/”]

Plugin Page

Amazon S3 Error Handling

AWS CloudFront with PHP

Create sub buckets on amazon s3

Amazon S3 Cloud HTML5 MP3 Player is not associated in any way to amazon.com

Amazon S3 Error Handling


Hits: 6139  

Amazon S3 PHP Class A standalone Amazon S3 (REST) client for PHP 5.2.x using CURL that does not require PEAR.

AWS Management Console Home
AWS Management Console Home

AMAZON S3 (Scalable Storage in the Cloud) PHP CLASS

AWS access info
AWS access info

// AWS access info
if (!defined(‘awsAccessKey’)) define(‘awsAccessKey’, ‘CHANGE-KEY’);
if (!defined(‘awsSecretKey’)) define(‘awsSecretKey’, ‘CHANGE-KEY’);

OO method (e,g; $s3->getObject(…)):
$s3 = new S3(awsAccessKey, awsSecretKey);

Statically (e,g; S3::getObject(…)):
S3::setAuth(awsAccessKey, awsSecretKey);

Warning: S3::getBucket(): [PermanentRedirect] The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.

We need to End Point for our S3 Buckets by default it’s s3.amazonaws.com

$s3 = new S3(awsAccessKey, awsSecretKey);

S3 Management Console
S3 Management Console

Else we need to specify here… s3-ap-southeast-1.amazonaws.com

S3 Management Console
S3 Management Console

$s3 = new S3(awsAccessKey, awsSecretKey, true, ‘s3-ap-southeast-1.amazonaws.com‘);

Warning: S3::getBucket(): [60] SSL certificate problem, verify that the CA cert is OK. Details: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

We need SSL enabled with CURL 😉

OpenSSL phpinfo()
OpenSSL phpinfo()
CURL With OpenSSL phpinfo()
CURL With OpenSSL phpinfo()

For class documentation see:
http://undesigned.org.za/files/s3-class-documentation/index.html

Recover MySQL root Password


Hits: 3109  

1. Stop the MySQL server.

# /etc/init.d/mysql stop

2. Start the MySQL server process with the –skip-grant-tables option so that it will not prompt for password.

# mysqld_safe –skip-grant-tables &

3. Connect to mysql server as the root.

# mysql -u root

mysql password
mysql password

Welcome to the MySQL monitor….
………..
Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the buffer.
mysql>

4. Setup new mysql root account password i.e. reset mysql password.

mysql> use mysql;
mysql> update user set password=PASSWORD(“NEW-PASSWORD”) where User=’root’;
mysql> flush privileges;
mysql> quit

5. Exit and restart the MySQL server.

# /etc/init.d/mysql stop

6. Start MySQL server.

# /etc/init.d/mysql start
# mysql -u root -p

 

 

SQL OR NOSQL?


Hits: 3914  

If you are searching for SQL or noSQL solutions, it is very difficult to decide among top SQL / noSQL Databases.

MySQL is the best for multi-document transactions, complex security, complex join, extreme compression.

NoSQL is good for linear scalability, high performance, schema flexibility.

But both depend on data need to manage 😉

Scalability

MySQL support master and slave approach for scaling (vertical scalability).
NoSQL supports horizontal scalability.

Vertical scalability is the ability to increase the capacity of existing hardware or software by adding resources – for example, adding processing power to a server to make it faster. On the other hand, horizontal scalability is the ability to connect multiple entities so that they work as a single logical unit.

Flexibility

SQL is the best for query flexibility, SQL supports both simple and complex queries.
MongoDB supports SQL query and indexes.

Down Time

NoSQL perform well for database size and server performance.

Transactions

SQL is king of transactions.
MongoDB and Redis support limited transactions.

Hosting

Every database solution need deployment, backups, monitoring, and performance tuning.

NoSQL are modern web-scale databases having qualities of non-relational, distributed, open-source and horizontally scalable database.

How to play wowza live video on ipad


Hits: 15791  

Install Wowza Examples

Take care for common problems:

Do you have the Application.xml in the right place?
[wowza-install-dir]/conf/[appName]/Application.xml

Server URL: rtmp://[wowza-address]/live
Stream Name: livestream

Which format need to broadcast from FMLE?
For iOS streaming, you’ll need h.264 baseline profile 3.0 on the video side and AAC on the audio side.

what is playlist.m3u8?
With Wowza cupertinostreaming…
http://[wowza-address]:1935/live/livestream/playlist.m3u8

Sample Code with HTML5 JWPlayer Support

<script type="text/javascript" src="jwplayer.js"></script> 

<div id='mediaplayer'></div>

<script type="text/javascript">  

jwplayer('mediaplayer').setup({
    'id': 'playerID',    'width': '623',    'height': '365',
    'provider': 'rtmp',  
    'streamer': 'rtmp://[wowza-address]/live',
    'autostart': 'true',
    'stretching': 'exactfit',  

    levels: [{
              bitrate: "500",
              file: "livestream",
              width: "800"
              }, {
              bitrate: "800",
              file: "livestream",
              width: "1280"
              }],   
               
    'modes': [
         {type: 'flash', src: 'player.swf'},
         {
          type: 'html5',
          config: {
           levels: [ {'file': 'http://[wowza-address]:1935/live/livestream/playlist.m3u8'} ],
           'provider': 'video'
          }
        }
    ]
  });
 
</script>

To play using Adobe Flash player (RTMP)

Server: rtmp://[wowza-address]/vod
Stream: mp4:sample.mp4

To play using RTSP/RTP player or device
rtsp://[wowza-address]:1935/vod/mp4:sample.mp4

To play using an Apple iDevice (Cupertino/Apple HTTP Live Streaming)

http://[wowza-address]:1935/vod/mp4:sample.mp4/playlist.m3u8

How to set up live streaming using an RTMP based encoder