Python bottle

Posted on 2022-06-16 | In programming language , Python

Use route to indicate path, 80 is the default port for HTTP.

import bottle

@bottle.route('/hello')
def hello():
    return "Hello World!"

bottle.run(host='localhost', port=80, debug=True)

Dynamic path V.S. static path. We can use filter such as <id:int> to convert input variables to certain type.

import bottle

@bottle.route('/hello/&lt;name>/&lt;address>')
def hello(name,address):
    return "Hello "+name

bottle.run(host='localhost', port=80, debug=True)

Use local resources. Pay special attention to the path. Local resource names should start with ‘/‘.

1
2
3

@bottle.route('/<filename:path>')
def server_static(filename):
  return bottle.static_file(filename, root='./resource/')

The default HTTP request method for route() is get(), we can use other methods post(), put(), delete(), patch(). The POST method is commonly used for HTML form submission. When entering URL address, GET method is used.

import bottle

def check_login(username, password):
    if username=='niuli' and password=='niuli':
        return True
    else:
        return False
    
@bottle.get('/login') # or @route('/login')
def login():
    return '''
        &lt;form action="/login" method="post">
            Username: &lt;input name="username" type="text" />
            Password: &lt;input name="password" type="password" />
            &lt;input value="Login" type="submit" />
        &lt;/form>
    '''

@bottle.post('/login') # or @route('/login', method='POST')
def do_login():
    username = bottle.request.forms.get('username')
    password = bottle.request.forms.get('password')
    if check_login(username, password):
        return "&lt;p>Your login information was correct.&lt;/p>"
    else:
        return "&lt;p>Login failed.&lt;/p>"

bottle.run(host='localhost', port=80, debug=True)

Return local files localhost/filename. Note path:path is important, otherwise the filename containing ‘/‘ may not be correctly recognized.

from bottle import static_file

	@route(’/<filepath:path>’)
	def server_static(filepath):
	    return static_file(filepath, root=’./resource’)

Use redirect to jump to another page: bottle.redirect('/login')

Use the HTML template. In the template file, the lines starting with % are python codes and others are HTML codes. We can include other templates in the current template by using % include('header.tpl', title='Page Title'). Cookies, HTTP header, HTML <form> fields and other request data is available through the global request object.

@bottle.route('/<name>')
@bottle.view('my_template.html')
def root(name="default"):
    return bottle.template('my_template', name=name)

@bottle.post('/login') # or @route('/login', method='POST')
def do_login():
    username = bottle.request.forms.get('username')
    password = bottle.request.forms.get('password')
    bottle.redirect('/'+username)

Some tricks for helping development
- bottle.debug(True) The default error page shows a traceback. Templates are not cached. Plugins are applied immediately.
- run(reloader=True) Every time you edit a module file, the reloader restarts the server process and loads the newest version of your code.

Fastest Way to Read Text, Image, and Video in Python

Posted on 2022-06-16 | In programming language , Python

When comparing the efficiency of different libraries, there may exist a few orders of magnitude difference. In the implementation in demand of high efficiency, locate the time-consuming function and replace it with the most efficient library function.

Text: Pandas
Installation: pip install pandas or conda install pandas

1 2	import pandas as pd data = pd.read_csv(text_name, sep=',', header=None)

Image: Pillow-SIMD, skimage, OpenCV, imageio

import cv2
import skimage
import imageio
from PIL import Image

#read a 1280x720 image
pil_im = Image.open(image_name) #0.0057s
pil_im = pil_im.resize((448,448)) #0.025s

sk_im = skimage.io.imread(image_name) #0.026s
sk_im = skimage.transform.resize(sk_im, (448, 448)) #0.060S

cv_im = cv2.imread(image_name) #0.021s
cv_im = cv2.resize(cv_im, (448, 448)) #0.0016s

im = imageio.imread(image_name) #0.033s

Pillow-SIMD is faster than Pillow, which is not reported here. OpenCV is the most efficient one here.

Video: OpenCV, skvideo, imageio

import cv2
import imageio
import skvideo

#read 30fps video with each frame 1280x720 
cap = cv2.VideoCapture(video_name)
ret, frame = cap.read() #0.002s

vid = imageio.get_reader(video_name, 'ffmpeg')
for image in vid.iter_data(): #0.004s

skvideo.setFFmpegPath(os.path.dirname(sys.executable))
videogen = skvideo.io.vreader(video_name)
for img in videogen: #0.073s

For OpenCV in Anaconda, it sometimes fails in reading from video but succeeds in reading from camera. In this case, /usr/bin/python is recommended. imageio and OpenCV are comparable here.

Anaconda

Posted on 2022-06-16 | In programming language , Python

python virtual environment: creat an isolated environment for each python version set, similar with pyenv.

Installation:

For Windows:

Anaconda navigator is a visualization tool for setting up environment and installing packages under each environment.

Python IDEs like Jupyter notebook and spyder can all be treated as packages under each environment.

For Linux:

bash Anaconda2-4.3.1-Linux-x86_64.sh

Environment

list: conda info —envs

create: conda create --name $newenv python=2.7 (if copy, use —clone $oldenv)

activate: source activate $newenv

revert: source deactivate

delete: conda remove —name $newenv —all

package

list: conda list

search: conda search pack

install new packages:

conda install pack
conda install pack=1.8.2 # install certain version
conda install —name env pack
search on http://anaconda.org/, no need to register
pip install pack
install from local pack

Tips: check available pythons: conda search —full-name python

MATLAB draw plot

Posted on 2022-06-16 | In programming language , MATLAB

% Set some nice settings.
grid on;
format long;

% Hold the graphics output until we are good to go.
hold all;

% To create some random test data.
x1 = 100 : 100 : 1000;
raw_y1 = [0.76,0.79,0.80,0.80,0.81,0.82,0.82,0.82,0.82,0.81];
raw_y2 = [0.85,0.81,0.83,0.83,0.82,0.79,0.78,0.80,0.82,0.82];
raw_y3 = [0.85,0.84,0.83,0.83,0.83,0.83,0.82,0.82,0.82,0.82];
legendText = cell(0);

plot(x1,y1,'--go','MarkerSize',5, 'MarkerFaceColor','g', 'LineWidth',3);
legendText(end+1) = { 'CUB' };

plot(x1,y2,':bo', 'MarkerSize',5, 'MarkerFaceColor','b', 'LineWidth',3);
legendText(end+1) = { 'SUN' };

plot(x1,y3,'-ro', 'MarkerSize',5,  'MarkerFaceColor','r', 'LineWidth',3);
legendText(end+1) = { 'Dogs' };

xlim([100 1000]);
ylim([0.72 0.90]);

set(gca,'fontsize',15)

% set sticks on x and y axis
get(gca, 'xtick');
set(gca, 'xtick', 100:100:1000);
get(gca, 'ytick');
set(gca, 'ytick', 0.72:0.02:0.90);

xlabel('# web training instances per category');
ylabel('Accuracy');

legend(legendText,'location','southeast');

hold off;

HTML+CSS

Posted on 2022-06-16 | In programming language , HTML

Website consists of three components: structure(HTML), representation(CSS), action(Javascript). XHTML is a strict version of HTML. The structure of webpage is previously done based on table, but currently via CSS.
Escape character in HTML source code: <:<, >:>,  :space
<p>:paragraph, <br>:break, <blockquote>:indented paragraph, <b>:bold, <i>:italic, <u>:underline, <s>:deleteline, <strong>:emphasize, <ul><li>:unordered list, <ol><li>:ordered list.
<a href="" target="_blank"></a>:hyperlink. <a href="mailto:ustcnewly@gmail.com"></a>. An area on the image can also be used to establish hyperlink based on <map> and <ared>.
Use <frameset> and <frame> to split webpage. Note <frameset> and <frame> belong to the the same level as <body>. <frameset cols="30%, 30%, *"><frame src=left.html>. <frame> is obsolete in HTML5.
<table border="1" cellpadding="4" cellspacing="6" ><tr><td></td><tr></tr></table>. Don’t miss any <tr> or <td>, otherwise the table will be messy. Use <colspan> or <rowspan> to merge cells. For professional table, we can use <caption>, <thead>, <tbody>, and <tfoot>. <table> is now rarely used for layout design.
Insert multimedia elements:
- image: <img src="a.jpg" height="200" width="200" alt="desc"/>
- flash: <embed src="a.swf" width="490" height="400" wmode="transparent" ></embed>
- bgmusic: <audio src="a.mp3" hidden="true" autoplay="true" loop="true"></audio>
- bgtile: <body background="a.png"></body>

Css: object, attribute, and value. Refer to official website for details.

h1{
    font-family:Calibri;
}

.myclass{
    font-family:Calibri;
}
&lt;p class="myclass">&lt;/p>

#myid{
    font-family:Calibri;
}
&lt;p id="myid">&lt;/p>
# one id name should be used only once in one html file.

Methods to use Css.
- a. Directly insert contents into <style></style>.
- b. <link href="mycss.css" type="text/css" rel="stylesheet">
- c. <style type="text/css">@import "mycss.css"</style>. One html file can import more than one Css files. A Css file can also import other Css files.
  
  The priority of a is higher than b/c. The more special, the higher priority. For the methods with equal priority, override principle is applied. The difference between b and c is that c loads all the Css codes while b only loads the corresponding Css code when necessary.
Some special usage of Css.
- class for specific tag: p.special{}
- union of multiple tags: h1,h2,p{}
- embeded tags: p span{} for descendents and p>span{} for child. complex embedding mixing tags, classes, and ids: td.out .inside strong{}. The easiest way to recognize this kind of embedding is “(tag)(.class) (tag)(#id)”
- consecutive tags: th+td+td+td{}, the style is applied to the third td.

Css formats for some commonly used tags.

body:

body{
    background-image:a.jpg;
    background-repeat:no-repeat;
    background-position:200px 100px;
	background-attachment:fixed;
	background-size: 20% 100%;
	cursor:pointer;
}

text:

p{	    
    float:left;
    font-family:Calibri;
    font-size:15px;
    font-style:italic;
    font-weight:bold;
    color:red;
    text-indent:2em;
    text-decoration:underline;
    text-transform:lowercase; #capitalize
    text-align:justify; 
    margin:5px 0px #top/down margin left/right margin
}

image:

img{
	float:left;
	border:1px #9999CC dashed;
	margin:5px; #margin-left/right/top/bottom
}

<div> box layout from inside to outside: content, padding, border, margin. Note width and height are for content instead of the whole box. margin can be assigned negative values. Assigning attribute values in clock-wise order. Missing values are equal to the values of opposite side. We should first understand the standard flow without the constraints of boxes.
- block-level (vertical arrangement): <ul>, <li>, <div>
- inline (horizontal arrangement): <strong>,<a>, <span>
  
  <div> and <span> are both blocks. <div> is vertial while <span> is horizontal. <span> can be used where no other proper tags can be used,s for instance, <span><img src="a.jpg"></span>.
  
  when using <div>, the margin between the top block and bottom block is max(top_block.bottom_margin,bottom_block.top_margin). when using <span>, the margin between the top block and bottom block is top_block.bottom_margin+bottom_block.top_margin.
floating box: Arrange the non-floating boxes first and then float the floating box to the left or right in the father content. The stuff in the non-floating boxes surrounds the floating box. For floating box: float:left. For non-floating box: clear:left.
box location: static(default), relative, absolute
- relative: position:relative; left:30px; top:30px. Relative shift from original position. Relative has no impact on father box and sibling box.
- absolute: position:absolute; top:0px; right:0px. Absolute coordinate in the nearest non-static ancestor. Other boxes treat this absolute box as non-existence.
<div style="display:inline"></div>, <span style="display:block"></span>, use display to modifiy the vertical or horizontal order. Set as none to make it invisible.

For hyperlink, a has pseudo classes:link, visited, hover, for example:

#navigation li a:link, #navigation li a:visited
{
	background-color:#1136c1;
	color:#FFFFFF;
}

Use div to split columns
- absolute method: left column uses absolute position (Note to change the position of father div to relative) and right column use left margin. The drawback is that bottom row will ignore the left column.
- float method: float left and float right. It is easy for fixed width or fixed ratio. For the mixed width such as 100%-30px, use the wrapper trick (negative margin).

JavaScript

<script>
var name ="newly"
document.write("my name is: "+name)
</script>

variable types

null: empty variable
string: parseFloat(string), parseInt(string), str.substring(0,3), str.slice(3,5), str.charAt(0), str.bold(), str.fontcolor("red"), str.length
number: Math.PI, Math.max, isNaN(value),
bool: true or false
Array: can be a mixture of numbers and strings, var a = new Array(10, 20, "newly")
1
2
3
4
for(n in actorAry)
{
document.write("<li>"+actorAry[n]);
}
myarr.toString(), myarr.join('-'), myarr.concat(tailarr), myarr.reverse(), myarr.sort(cmpfunc), slice, splice

stack&&queue operations: myarr.push("newstr"), myarr.pop(), myarr.shift()(dequeue), myarr.unshift()(pushfront)

Structure: There is no concept: class. Use function to construct an object. var mycard = Card("newly", 20), showCardInof.call(mycard,arg1,arg2) or mycard.showCardInfo(arg1,arg2), mycard=null

function showCardInfo(arg1,arg2)
{
    document.write(this.owner);
}
function Card(owner, rate)
{
    this.owner = owner;
    this.rate = rate;
    this.showCardInfo=function(arg1,arg2)
    {
        document.write(this.owner);
    }
}

type related: nameList instanceof Array, typeof("newly")

commonly used built-in functions or classes
- alert dialogue: alert("msg")
- input diaglogue: var age=prompt("Input your age", "0")
- confirm dialogue: confirm("Are you sure?")
- Date and time:
  1
  2
  3
  var cur = new Date();
  cur.setYear(2016);
  var seconds = cur.getSeconds();
- Error:
  1
  2
  var e = new Error();
  document.write(e.number&0xFFFF+e.description);

operate on the html elements

<img src="./images/web_fig.jpg" width=500px>

The whole HTML page is a DOM tree.

Visit the DOM elements

document.myform.elements[0].value  //myform is the form name
document.getElementById("myid").childNodes[0].nodeName/nodeValue/nodeType
document.getElementById("myid").childNodes[0].getAttribute("attr")/setAttribute("attr","attval")	
document.getElementsByTagName("tag")	
document.getElementsByName("name")

Since id is unique, we use single form element for Id, but plural form elements for Name and TagName.

Change the DOM elements

var docBody = document.getElementById("DocBody");
var imgObj = document.createElement("<img>");
var newTextNode = document.createElement("content");
imgObj.src = url;
docBody.appendChild(imgObj);

use Javascript to modify the CSS style, document.vlinkColor, document.linkColor, document.alinkColor, document.bgColor, document.fgColor, document.body.text, name1.vspace, name1.hspace.
event handling: we can set event triggering function for certain element or a class of elements.
- commonly used events (case insensitive): onblur, onchange, onclick, onfocus, onload, onmouseover, onmouseout, onmousedown, onmouseup, onselect, onsubmit, onunload. Form-related events: myform.onReset, myform.onSubmit, myform.action="mailto:ustcnewly@gmail.com"
- three approaches to trigger event function
  a) directly embed function scripts: onlick="javascript:alert("msg")" //javascript: can be eliminated
  b) call the event function: onkeyup = "OnKeyUp()"
  1
  2
  3
  4
  5
  6
  7
  8
  function OnKeyUp(_e)
  {
  var e = _e?_e:window.event;
  if (event.keyCode==13)
  {
  alert("Your input is "+Text1.value);
  }
  }
  c) set the event attribute for certain elements:
```
   <script for="document" event="oncontextmenu"> 
   window.event.returnValue = false;
   document.oncontextmenu = hideContextMenu2;
   </script>;
```
time control:to=window.setTimeout("func()",3000), clearTimeout(to), tm=setInterval("func()",1000), clearInterval(tm)
redirect href location or set anchor point:window.location.href="www.baidu.com", <a name="anchor1"></a>
history: history.back(), history.forward(), history.go(n), n>0 means go back n pages, otherwise go forward n pages.
cookie contains key-value pairs: document.cookie="user=newly;passwd=hehe;"

Call C++ in Python

Posted on 2022-06-16 | In programming language , cross-language

1.ctype

similar as in Python_call_C.md
add extern “C” before each function, otherwise an error ‘undefined symbol’ will be thrown.

However, something might go wrong on terms with complicated Class.

2.Use Boost.Python

more powerful and complicated

Call C in Python

Posted on 2022-06-16 | In programming language , cross-language

C: build a library

1 2	gcc -c -fPIC libtest.c gcc -shared libtest.o -o libtest.so

Python: load the library

1
2
3

from ctypes import *
libtest = cdll.LoadLibrary(libtest.so')
print libtest.func(arg_list)

Makefile sample and python sample code

c_void_p

For the types without need to know the detailed layout, we can just use c_void_p, especially when we cannot find the strictly matched self-defined type such as LP_cfloat_1024.

pointer, POINTER, byref

POINTER is used for defining type.

pointer and byref function similarly. However, pointer does a lot more work since it constructs a real pointer object, so it is faster to use byref if you don’t need the pointer object in Python itself.

memory issue:

write free function in C code

void freeme(char *ptr)
{
    printf("freeing address: %p\n", ptr);
    free(ptr);
}

call free function in Python

lib = cdll.LoadLibrary('./string.so')
lib.freeme.argtypes = c_void_p,
lib.freeme.restype = None
lib.freeme(ptr)

Caffe

Posted on 2022-06-16 | In programming language , C++

Fundamental Stuff:

Key Notes:

Required Packages:

CUDA
OpenCV
BLAS: (Basic Linear Algebra Subprograms)
operations like matrix multiplication, matrix addition,
both implementation for CPU(cBLAS) and GPU(cuBLAS).
provided by MKL(INTEL), ATLAS, openBLAS, etc.
Boost: a c++ library, use some of its math functions and shared_pointer.
glog,gflags:provide logging & command line utilities. Essential for debugging.
leveldb lmdb: database io for your program. Need to know this for preparing your own data.
protobuf: an efficient and flexible way to define data structure. Need to know this for defining new layers.

Boost_tokenizer

Posted on 2022-06-16 | In programming language , C++

1	#include< boost/tokenizer.hpp>

split string: default space

string s = "This is,  a test";  
tokenizer<> tok(s);  
for(tokenizer<>::iterator beg=tok.begin(); beg!=tok.end();++beg)
{  
   cout << *beg << "\n";  
}

split string: drop delimiter

1
2
3

typedef boost::tokenizer<boost::char_separator<char>>  tokenizer;
boost::char_separator< char> sep("-;|");
tokenizer tokens(str, sep);

split string: drop delimiter and keep delimiter

1
2
3

typedef boost::tokenizer< boost::char_separator< char> > tokenizer;
boost::char_separator< char> sep("-;", "|", boost::keep_empty_tokens);
tokenizer tokens(str, sep);

special kinds of tokenizer

1
2
3

int offsets[] = {2,2,4};   //three segment length  
offset_separator f(offsets, offsets+3);  
tokenizer< offset_separator> tok(s,f);

Boost_regex

Posted on 2022-06-16 | In programming language , C++

1 2	#include < boost/regex.hpp> using namespace boost;

regex_match

regex expression("^select ([a-zA-Z]*) from ([a-zA-Z]*)");
std::string in="select name from table";
cmatch what;
if(regex_match(in.c_str(), what, expression))
{

	for(int i=0;i< what.size();i++)
	{
		cout<< what[i].first<< "|"<<what[i].second<< endl;
		cout<< "str :"<< what[i].str()<< endl;
	}
}

regex_search

boost::regex r("(a+)");
string content = "bbbaaaaacccaaaaddddaaaeeeaaa";
boost::smatch m;
string::const_iterator strstart = content.begin();
string::const_iterator strend = content.end();
while(boost::regex_search(strstart, strend, m, r))
{
    //0 for the whole, m[i] is the i-th group
    cout << m[0] << endl;
    //m[i].first and m[i].second stands for the begin idx and end idx
    strstart = m[0].second;
}

Note regex_match(in.c_str(), what, expression) is for complete match while regex_match(in.c_str(), what, expression) is for incomplete match.
regex_match(in.c_str(), expression, target_exp).

special setting for regex

1	boost::regex e2(my_expression, boost::regex::icase); \\case insensitive