(https://github.com/soumith/cvpr2015/blob/master/Deep%20Learning%20with%20Torch.ipynb)
(https://github.com/soumith/cvpr2015/blob/master/Char-RNN.ipynb)
(http://tylerneylon.com/a/learn-lua/)
num = 42 -- All numbers are doubles.
t = nil -- Undefines t; Lua has garbage collection.
while num ## 50 do
num = num + 1 -- No ++ or += type operators.
end
if num ## 40 then
print('over 40')
elseif s ~= 'walternate' then -- ~= is not equals.
-- Equality check is == like Python; ok for strs.
io.write('not over 40\n') -- Defaults to stdout.
else
-- How to make a variable local:
local line = io.read() -- Reads next stdin line.problem: fail to log into gui. /dev/nvidia#### has nothing.
solved by not install opengl
(https://github.com/soumith/cvpr2015/blob/master/NNGraph%20Tutorial.ipynb)
function get_rnn(input_size, rnn_size)
-- there are n+1 inputs (hiddens on each layer and x)
local input = nn.Identity()()
local prev_h = nn.Identity()()
-- RNN tick
local i2h = nn.Linear(input_size, rnn_size)(input)
local h2h = nn.Linear(rnn_size, rnn_size)(prev_h)
local added_h = nn.CAddTable()({i2h, h2h}) --performs an element-wise addition
local next_h = nn.Tanh()(added_h)
nngraph.annotateNodes()
return nn.gModule({input, prev_h}, {next_h})
end(https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearning/practicals/practical5.pdf)
(https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearning/practicals/practical6.pdf)
(https://github.com/oxford-cs-ml-2015/practical6)
--the model can be found at https://en.wikipedia.org/wiki/Long_short-term_memory
function LSTM.lstm(opt)
local x = nn.Identity()()
local prev_c = nn.Identity()()
local prev_h = nn.Identity()()
function new_input_sum()
-- transforms input
local i2h = nn.Linear(opt.rnn_size, opt.rnn_size)(x)
-- transforms previous timestep's output
local h2h = nn.Linear(opt.rnn_size, opt.rnn_size)(prev_h)
return nn.CAddTable()({i2h, h2h})
end
local in_gate = nn.Sigmoid()(new_input_sum())
local forget_gate = nn.Sigmoid()(new_input_sum())
local out_gate = nn.Sigmoid()(new_input_sum())
local in_transform = nn.Tanh()(new_input_sum())
local next_c = nn.CAddTable()({
nn.CMulTable()({forget_gate, prev_c}),
nn.CMulTable()({in_gate, in_transform})
})
local next_h = nn.CMulTable()({out_gate, nn.Tanh()(next_c)})
return nn.gModule({x, prev_c, prev_h}, {next_c, next_h})
end
return LSTM1.description:this task uses news text to produce train data. The source sentences are cleaned sentences from news documents and the target sentences are the next sentence after the source sentence.
2.delete sentences of length <5 or >25
sudo pip install cython
sudo apt-get install libhdf5-dev
sudo pip install h5py
follow instructions on torch.ch
(https://github.com/torch/nn/blob/master/doc/table.md)
(https://github.com/torch/torch7/blob/master/doc/tensor.md)
(https://github.com/torch/torch7/blob/master/doc/maths.md)
##2016-11-18
(https://github.com/torch/torch7/wiki/Cheatsheet)
(http://hunch.net/~nyoml/torch7.pdf)
(http://www.wildml.com/2016/01/attention-and-memory-in-deep-learning-and-nlp/)
after diable nouveau driver and reboot the system, cannot enter into text mode.
Neural Turing Machines:https://arxiv.org/pdf/1410.5401v2.pdf
minimum number of editing steps to transform output to reference.
-
match: words match, no cost
-
substitution: replace one word with another
-
insertion: add word
-
deletion: drop word
-
WER=(substitutions+insertions+deletions)/reference-length
-
bleu:n-gram overlap between machine translation output and reference translation
(https://arxiv.org/pdf/cs/0112005v1.pdf)
1.Synonyms:
Original: 65 is the traditional age for workers to retire in the U.S.
Paraphrase: 65 is the traditional age for employees to retire in the U.S.
2.Condensation:
Original: 65 is the traditional age for workers to retire in the U.S.
Paraphrase: 65 is the traditional retirement age in the U.S.
3.Circumlocution
Original: 65 is the traditional age for worker to retire in the U.S.
Paraphrase: 65 is the traditional age for workers to end their professional career in the U.S.
4.Phrase Reversal
Original: 65 is the traditional age for workers to retire in the U.S.
Paraphrase: In the U.S., the traditional age for workers to retire is 65.
5.Active-Passive Voice
Original: The company fired 15 workers.
Paraphrase: 15 workers were fired by the company.
6.Alternate Word Form
Original: A manager’s success is often due to perseverance.
Paraphrase: A manager often succeeds because of perseverance. Managers’ success is often because they persevere.
when attn=0 use the hidden state of the last rnn unit as the context vector.
read GAN tutorial
(http://web.stanford.edu/class/cs224u/materials/cs224u-2016-bowman.pdf) slide:23
-
equivalence
-
forward entailment
-
reverse entailment
-
negation
-
alternation
-
cover
-
independence
##2016-12-19
Ideas: words that tend to occur in the same contexts tend to have similar meanings.
Ideas: each filter of the CharCNN is essentially learning to detect particular character n-grams.
read 中文信息处理发展报告
read Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks
####繁简转换
opencc -i wiki.zh.text -o wiki.zh.text.jian -c zht2zhs.ini
The punctuation comma in Chinese sometimes functions as a period causing the diffuculty of segmenting sentences in a paragraph.
read text segmentation
IOB tokenization
Label tokens of a sentence.
sudo apt-get update
sudo apt-get install git-core curl zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev python-software-properties libffi-dev
cd
git clone https://github.com/rbenv/rbenv.git ~/.rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
exec $SHELL
git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build
echo 'export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"' >> ~/.bashrc
exec $SHELL
rbenv install 2.3.3
rbenv global 2.3.3
ruby -v
#optional:
echo "gem: --no-ri --no-rdoc" > ~/.gemrc
read The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems
Retrieve-based chatbot. Use dual encoders to predict whether a context and a response is a match.
read Show and Tell
find a repo which is able to convert pdf into html (https://github.com/coolwanglu/pdf2htmlEX)
magic installation:(https://gist.github.com/rajeevkannav/d07f822e209a22d07176)
(https://docs.docker.com/engine/installation/linux/ubuntulinux/)
sudo apt-get update
sudo apt-get install apt-transport-https ca-certificates
sudo apt-key adv \
--keyserver hkp://ha.pool.sks-keyservers.net:80 \
--recv-keys 58118E89F3A912897C070ADBF76221572C52609D
echo "deb https://apt.dockerproject.org/repo ubuntu-trusty main" | sudo tee /etc/apt/sources.list.d/docker.list
sudo apt-get update
sudo apt-get install linux-image-extra-$(uname -r) linux-image-extra-virtual
sudo apt-get update
sudo apt-get install docker-engine
sudo service docker start
sudo docker run hello-world
tar -zxvf apache-tomcat-7.0.73.tar.gz
sudo mv apache-tomcat-7.0.73 /opt/
cd /opt/apache-tomcat-7.0.73/bin/
sudo gedit setclasspath.sh
//add the following text to setclasspath.sh in the beginning
//export CATALINA_HOME=/opt/apache-tomcat-7.0.73
//export JAVA_HOME=/usr/local/java/jdk1.7.0_80
sudo ./startup.sh
//Using CATALINA_BASE: /opt/apache-tomcat-7.0.73
//Using CATALINA_HOME: /opt/apache-tomcat-7.0.73
//Using CATALINA_TMPDIR: /opt/apache-tomcat-7.0.73/temp
//Using JRE_HOME: /usr/lib/jdk/jdk1.7.0_80
//Using CLASSPATH: /opt/apache-tomcat-//7.0.73/bin/bootstrap.jar:/opt/apache-tomcat-7.0.73/bin/tomcat-juli.jar
//Tomcat started.
sudo ./shutdown.sh
//Using CATALINA_BASE: /opt/apache-tomcat-7.0.73
//Using CATALINA_HOME: /opt/apache-tomcat-7.0.73
//Using CATALINA_TMPDIR: /opt/apache-tomcat-7.0.73/temp
//Using JRE_HOME: /usr/lib/jdk/jdk1.7.0_80
//Using CLASSPATH: /opt/apache-tomcat-//7.0.73/bin/bootstrap.jar:/opt/apache-tomcat-7.0.73/bin/tomcat-juli.jar
Useful blog(http://blog.csdn.net/xuzhongxiong/article/details/52717285)
sudo apt-get install build-essential
sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
cd opencv-2.4.11
gedit modules/gpu/src/graphcuts.cpp
replace line 45 with "#if !defined (HAVE_CUDA) || defined (CUDA_DISABLER) || (CUDART_VERSION>=8000)"
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D CUDA_GENERATION=Kepler ..
make -j8
sudo make install
#sudo /bin/bash -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf'
#sudo ldconfig
sudo apt-get install ant
#export JAVA_HOME=/usr/lib/jvm/java-6-oracle
cmake -DBUILD_SHARED_LIBS=OFF ..
make -j8
sudo make install
-
open "file/project structure/dependencies", add opencv jar.
-
open "file/project structure/libraries/+/java", add opencv jar.
-
in vm option, write: -Djava.library.path=/your_opencv_path/release/lib
-
add "System.loadLibrary(Core.NATIVE_LIBRARY_NAME);" in java file
sudo apt-get update
sudo apt-get install nvidia-367
sudo apt-get install mesa-common-dev
sudo apt-get install freeglut3-dev
sudo reboot
sudo apt-get install g++
sudo service lightdm stop
#don't install driver in the next step!!!
sudo sh cuda_8.0.44_linux.run
sudo reboot
sudo gedit /etc/profile
source /etc/profile
uname -r
nvcc -V
tar xvzf cudnn-8.0-linux-x64-v5.1-ga.tgz
sudo cp -P cuda/include/cudnn.h /usr/local/cuda-8.0/include
sudo cp -P cuda/lib64/libcudnn* /usr/local/cuda-8.0/lib64
sudo chmod a+r /usr/local/cuda-8.0/include/cudnn.h /usr/local/cuda-8.0/lib64/libcudnn*
sudo apt-get install python-pip python-dev
export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-0.12.1-cp27-none-linux_x86_64.whl
sudo pip install --upgrade $TF_BINARY_URL
sudo gedit /etc/shiny-server/shiny-server.conf
change "run_as shiny" to "run_as username"
sudo ufw allow 3838/tcp
-
Follow instructions (https://my.oschina.net/u/1010578/blog/390094)
-
Set artifact (http://blog.csdn.net/petershusheng/article/details/52382216)
-
Add jsp-api.jar and servlet-api.jar
Read Question Answering
Why QA? What kind of questions should be answered by a domain specific QA system?
Read Stanford QA slides
Read StalemateBreaker: A Proactive Content-Introducing Approach to Automatic Human-Computer Conversation
sudo add-apt-repository ppa:kirillshkrogalev/ffmpeg-next
sudo apt-get update
sudo apt-get install ffmpeg
- Distributional Hypothesis: Words that occur in similar contexts tend to have similar meanings
- Extended Distributional Hypothesis: Patterns that co-occur with similar pairs tend to have similar meanings.
- Latent Relation Hypothesis: Pairs of words that co-occur in similar patterns tend to have similar semantic relations.
- a multi-instance learning assumption (Dietterich et al., 1997) that two sentences under the same topic (we highlight topics in bold) are paraphrases if they contain at least one word pair.
Met a problem when creating a project by scrapy.
AttributeError: 'module' object has no attribute 'OP_NO_TLSv1_1'
Solution:
sudo pip install openssl
sudo pip install --upgrade pyOpenSSl
copy random N files to a directory
ls | shuf -n 11 | xargs cp -t /home/han/Desktop/
"Sum-squared error also equally weights errors in large boxes and small boxes. Our error metric should reflect that small deviations in large boxes matter less than in small boxes. To partially address this we predict the square root of the bounding box width and height instead of the width and height directly."
#rename all files in a directory
n=1
for f in `ls ./`;do mv $f $n.jpg; echo $n; n=$[$n+1]; done
mydir = check
mkdir $mydir
cat result.txt | awk -F " " '{if($2<="0.5") print $0}' > prob_less_50.list
n=1;for f in `cat prob_less_50.list`;do cp $f ./$mydir/$n.jpg; echo $n; n=$[$n+1]; done
ls ./mydir > rest.txt
python sel.py prob_less_50.list rest.txt todel.txt
cat todel.txt norpl.txt > negd.txt
python del.py poss2.txt todel.txt poss3.txt
python del.py xg1.txt todel.txt xg2.txt
Read Mask R-CNN
check disk space
df -h
use Kaldi
#install Kaldi
git clone https://github.com/suzhoushr/kaldi-shr.git
cd kaldi-shr/
chmod +x -R extras/
chmod +x configure
cd ./tools
#tar -xvzj openfst-1.3.4.tar.gz
./extras/check_dependencies.sh
#sudo apt-get install subversion
sudo ln -s -f bash /bin/sh
make -j4
cd ../src
./configure
make -j4
#cd ../nnetbin/
#make -j4
#preprocess data
./shr_scripts/chw/utils/img2condata.py train.list "train" | copy-feats ark:- ark,scp:./data/data.tr.ark,./data/data.tr.scp
./shr_scripts/chw/utils/img2condata.py val.list "test" | copy-feats ark:- ark,scp:./data/data.cv.ark,./data/data.cv.scp
deploy war package in tomcat. encountered a problem "java.lang.UnsatisfiedLinkError: no segmentor_jni in java.library.path"
solved by
sudo gedit /opt/apache-tomcat-7.0.78/bin/catalina.sh
\#write
JAVA_OPTS=" -Djava.library.path=/home/han/Software/ltp4j/libs"
compile ltp4j on centos
cmake -DLTP_HOME=`pwd`/ltp/ .
make
compile java from shell
javac -classpath /path/to/jar test.java
java -Djava.ext.dirs=/path/to/jar/ test
train a language model using irsltm
/disk03/czj_workspace/irstlm/build/bin/build-lm.sh -i data_ltp.txt -n 4 -f 3,3,2,2 -k 10 -v -o train.irstlm.gz
\#check tomcat process id
ps -ef | grep tomcat
\#check tomcat log
tail -100f ./logs/catalina.out
zip and upload file at the same time
rsync -z -P data_ltp.txt -e "ssh -p 59522" root@###.###.##.###:/data/czj_workspace
cat data_sentence.txt | sed 's/[ ]*[,，][ ]*/，\n/g' | awk '{if(length($0)>5) print}' | grep -E "大有.*之势" | less
Methods for addressing data imbalance
read paper abstractions on NIPS 2016
gibbs sampling, scan order, random scan, systematic scan.
ADMM-net, to reconstruct MRI data from a small number of under-sampled data
Bregman theorem
swapout, a regularization method, combination of dropout and stochastic depth.
introduced rado loss for classification problems
as the name of the paper suggests.
provide fast and competitive seedings for k-Means clustering without prior assumptions on the data
learn about physical object motion without labels
generate 3D objects
use a single input frame to generate future frames.
Human decision-making practically is not optimal under constraints such as time.
Boosting CNN for facial action recognition.
a bayesian treatment for NN, to alleviate the problem of insufficient data.
existing object detection algorithms ignore interdependency among different objects and deviate from the human perception procedure
transfer learning
mutli-armed bandit?
predict pixel depth in a image.
distinguish distributions
Test the independence of two random vectors, convert multivariate test to univariate test by imposing a distance measure.
Sythesize images using GAN given what and where to draw.
The Sound of APALM Clapping: Faster Nonsmooth Nonconvex Optimization with Stochastic Asynchronous PALM
a block coordinate stochastic proximal-gradient method for solving nonconvex, nonsmooth optimization problems
multi-task learning
compressing CNN