ERROR 1071 (42000): Specified key was too long; max key length is 767 bytes

If Any field is greater than 255 eg varchar(256) then reduce them to 255 and it should work.It seems 255 is the limitation.

Installing Nginx on Mac using Brew

Install Using Brew . ( Don’t have brew ?? go Here )

• Brew install nginx 
• Nginx will be installed and started by brew itself
• If not then type sudo nginx
• Test nginx by opening this url in browser : – http://localhost:8080

 

 

 

Import CSV to mysql

To import an Excel file into MySQL, first export it as a CSV file. Remove the CSV headers from the generated CSV file along with empty data that Excel may have put at the end of the CSV file.

You can then import it into a MySQL table by running:

load data local infile 'uniq.csv' into table tblUniq fields terminated by ','
enclosed by '"'
lines terminated by '\n'
(uniqName, uniqCity, uniqComments)

 

 

 

 

mysql retrieve first 100 characters

 

SELECT LEFT(field, 100) AS excerpt FROM table(s) WHERE ...

See the LEFT() function

 

Calculate n-gram in python

def ngrams(sequence, n, pad_left=False, pad_right=False, pad_symbol=None): 
	if pad_left: 
		sequence = chain((pad_symbol,) * (n-1), sequence) 
	if pad_right: 
		sequence = chain(sequence, (pad_symbol,) * (n-1)) 
	sequence = list(sequence)     
    count = max(0, len(sequence) - n + 1) 
    return [tuple(sequence[i:i+n]) for i in range(count)] 

Now lets get Started:

1) Bigrams:
sequence = [1,2,3,4,5]
n=2
ngrams(sequence,n)
[(1,2),(2,3),(3,4),(4,5)]

For Trigrams n=3 and so on..
If you want a left or right padding when the numbers are odd then mention the left Padding or right padding variable

Enjoy.

How to grep inside a folder

This is how we do it

grep -nr string my_directory

Decoding http response with gzip encoding in python

Here is the function to use:

import gzip
import StringIO
import zlib

def decode (page):
    encoding = page.info().get("Content-Encoding")    
    if encoding in ('gzip', 'x-gzip', 'deflate'):
        content = page.read()
        if encoding == 'deflate':
            data = StringIO.StringIO(zlib.decompress(content))
        else:
            data = gzip.GzipFile('', 'rb', 9, StringIO.StringIO(content))
        page = data.read()

    return page

Twitter oauth2 application only using python

Here is the first method using urllib library

import urllib
import base64
import httplib&lt;/code&gt;</pre>

CONSUMER_KEY = 'my_key'
CONSUMER_SECRET = 'my_secret'
encoded_CONSUMER_KEY = urllib.quote(CONSUMER_KEY)
encoded_CONSUMER_SECRET = urllib.quote(CONSUMER_SECRET)

concat_consumer_url = encoded_CONSUMER_KEY + ":" + encoded_CONSUMER_SECRET

host = 'api.twitter.com'
url = '/oauth2/token/'
params = urllib.urlencode({'grant_type' : 'client_credentials'})
req = httplib.HTTPSConnection(host)
req.putrequest("POST", url)
req.putheader("Host", host)
req.putheader("User-Agent", "My Twitter 1.1")
req.putheader("Authorization", "Basic %s" % base64.b64encode(concat_consumer_url))
req.putheader("Content-Type" ,"application/x-www-form-urlencoded;charset=UTF-8")
req.putheader("Content-Length", "29")
req.putheader("Accept-Encoding", "utf-8")

req.endheaders()
req.send(params)
resp = req.getresponse()

print resp.status, resp.reason
print resp.read()

This is the Second Method using pycurl

import sys # used for the storage class
import pycurl # used for curling
import base64 # used for encoding string
import urllib # used for enconding
import cStringIO # used for curl buffer grabbing
import json # used for decoding json token
import time # used for stuff to do with the rate limiting
from time import sleep # used for rate limiting
from time import gmtime, strftime # used for gathering time

def get_bearer_token(consumer_key,consumer_secret):
	# enconde consumer key
	consumer_key = urllib.quote(consumer_key)
	# encode consumer secret
	consumer_secret = urllib.quote(consumer_secret)
	# create bearer token
	bearer_token = consumer_key+':'+consumer_secret
	# base64 encode the token
	base64_encoded_bearer_token = base64.b64encode(bearer_token)
	# set headers
	headers = [
	"POST /oauth2/token HTTP/1.1",
	"Host: api.twitter.com",
	"User-Agent: shobhit Twitter Application-only OAuth App Python v.1",
	"Authorization: Basic "+base64_encoded_bearer_token+"",
	"Content-Type: application/x-www-form-urlencoded;charset=UTF-8",
	"Content-Length: 29"]
	token_url = "https://api.twitter.com/oauth2/token"
	buf = cStringIO.StringIO()
	access_token = ''
	pycurl_connect = pycurl.Curl()
	pycurl_connect.setopt(pycurl_connect.URL, token_url) # used to tell which url to go to
	pycurl_connect.setopt(pycurl_connect.WRITEFUNCTION, buf.write) # used for generating output
	pycurl_connect.setopt(pycurl_connect.HTTPHEADER, headers) # sends the customer headers above
	pycurl_connect.setopt(pycurl_connect.POSTFIELDS, 'grant_type=client_credentials')
        pycurl_connect.perform() # perform the curl
	access_token = buf.getvalue() # grab the data
	pycurl_connect.close() # stop the curl
	x = json.loads(access_token)
	bearer = x['access_token']
	return bearer

Library not loaded: libmysqlclient.16.dylib error

Try making this symbolic link and it should work

sudo ln -s /usr/local/mysql/lib/libmysqlclient.18.dylib /usr/lib/libmysqlclient.18.dylib

#################################################################################
PS: Check the libmysqlclient path which can be done via locate libmysqlclient

Permission Error PEM File while doing ssh at Amazon EC2

If you are getting error like this

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: UNPROTECTED PRIVATE KEY FILE! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0444 for '/Users/amazon.pem' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
bad permissions: ignore key: /Users/amazon.pem
Permission denied (publickey).

then you need to do is this:

chmod 400 mykey.pem

Taken from Amazon’s instructions –

Your key file must not be publicly viewable for SSH to work. Use this command if needed: chmod 400 mykey.pem