shell test部分

[hujianglang]

zsh:
Terminal:(emulator) :emulate a (texted-based)terminal inside the GUI environment
SSH to server
Running sshd: daemon of SSH server
strong password or use ssh key to login
keep the connection:
tmux ,
screen,
etc.

keyboard shortcuts
ctrl+r ( to find history)
tab (to autofill)
ctrl+c (to kill SIGINT)
Build from source (no suitable version, or need to modify their code)
README/INSTALL doc
configure and make install

ls | grep "build"
pipe: | use the stdout of previous command as the stdin of the next
command1 > file
command2 < file

ls > ls_out
grep build < ls_out

question1:
#include <stdio.h>
int main(void)
{
printf("stdout\n");
fprintf(stdout,"stdout\n");
fprintf(stderr, "stderr\n");
return 0;
}

./test > test_out
stderr
./test > test_out 2>test_err_out
cat test_err_out
stderr
2表示错误时候重定向输出到其他文件中

其他必须掌握的命令:
Basic Tools(commands)
File: touch , cp ,rm , cat , find , head , tail , less,(列出内容然后可以利用vim等工具翻看），mkdir, ln
simple functions: sort, wc
How to use?
-help, --help
man [command]

以下两个有例子可参考：
通过这两个例子可以学习shell example
https://command-not-found.com/
TLDR: https://tldr.sh

找具体的关键字：
ag command(also support regex and stdin (pipe , from stdout of other command)
Usage Scenario : Find keyword in code, doc, stdout, etc.
比如查找目录下代码中的某个函数在目录下的文件中的位置等：
ag xxfunction()
ps -A | ag sshd (查找当前在运行的sshd)

awk (usage scenario: Result (data) Processing
-Domain-specific language designed for text processing(c-like)
-Typically used as a data extraction and reporting tool

Normal use cases:
Average, max, min
Get data in a certain column
simple conditional logic

awk:
Usage scenario: Result (data) processing:
Example：Grab Data from a certain column
example1:
cat tmp
1 a
2 b
3 c
4 d
5 f

$1 表示第一列
$2 表示第二列

提取第二列:
cat tmp | awk '{print $2}'
tips:awk 会逐行打印，但是指定了$2 因此只会打印第二列
execute this code each line

example2:Average
cat tmp | awk 'BEGIN {cnt=0} {sum+=$1;cnt+=1} END {print (sum/cnt)}'

awk
Usage Scenario: Result(data) Processing
Example:Conditional Logic
cat tmp | awk '{if($1>3) print $2}'
A lot more can be done with this simple tool
Use your imagination!

cat tmp
1 a
2 b
3 c
4 d
5 e

sed
Usage Scenario: Result(data) Processing
. Edit text in a scriptable manner
Example : Get a certain line from a file:
sed -n '3 p' ./tmp

shell脚本使用：
bash ./run.sh arg1 arg2 arg3
脚本1：
rand.sh
echo $RANDOM

cat measure.sh
file=$1
awk 'BEGIN{ cnt=0} {sum+=($1); cnt+=1} END {print (sum/cnt)}' $file

example#1
Running experiments Multiple Times and Get the Average Result:
Dummy Experiment Output
Throughout 10592 ops/s
sed get the second line
awk get the number

run_dummy.sh
exp_times=100
result_file=tmp_result
echo "" > $result_file
for i in seq 1 $exp_times
do
bash ./run_exp.sh | sed -n '2 p' | awk '{print $2}' >> $result_file
#sed 提取第二行
done
awk 'BEGIN {cnt=0} {sum+=$1;cnt++;} END {print sum/cnt}' $result_file
Run under different configuration and use gnuplot to plot

vim ./rand_req.sh
echo $RANDOM

run_req.sh:
for i in seq 0 1000
do
bash ./rand_req.sh
done

run_exp.sh:
bash ./run_req.sh > req_time

$bash ./run_exp.sh

cat tmp
111
222
33123
23

sort -g ./tmp
example:
resultfile=$1
sort -g $resultfile > $resultfile-sort

wc -l ./req_time-sort

#htop command
#wc : work count
example2：
resultfile=$1
sort -g $resultfile > $resultfile-sort
lineno=wc -l $resultfile | awk '{print $1}'
#echo $lineno
p99_line=$[$lineno * 99 / 100]
#echo $p99_line

sed -n "$p99_line p" $resultfile-sort
awk -v lineno=$lineno 'BEGIN {cnt=0} {if(cnt % 2 == 0) print(cnt/lineno" " $1)
cnt++}' $resultfile-sort > $resultfile-cdf

Also read & finish:
https://missing.csail.mit.edu/2020/course-shell/
https://missing.csail.mit.edu/2020/shell-tools/