burpsuite

scope

排除的scope

1
.*\.(huawei|vmall|google|googleapis|gvt2|gstatic|googleusercontent|google-analytics|baidu|wappalyzer)\.com

隐藏的ext

1
woff,woff2,gif,jpg,png,svg,jpeg,css,ico,ttf,mp3,mp4,webm

linux

crontab的使用

参考 https://askubuntu.com/a/216711 ,每个用户有自己的crontab,如bob用户的位于 /var/spoll/cron/crontabs/bob ,但是不建议直接编辑该文件。而应该通过crontab -e来编辑。 也可以用crontab -l > backup ,然后编辑backup文件,再 crontab backup 来导入,就会和 crontab -e 编辑执行一样的语法检查。

bash命令、语法和脚本编写

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
##############################################################
### Linux是伟大的发明,尝试用golang实现bash的功能是浪费生命。 ###
##############################################################
#-------------------------------------------------------------#
# 算数表达式
$((123 + 1))
#-------------------------------------------------------------#
# 删除字符串的某个子字符串
tr -d '\r\n'
#-------------------------------------------------------------#
# 如果用alias,右侧命令用单引号包裹,免得出现奇怪的问题。
#-------------------------------------------------------------#
# alias无法传参,需传参的场景优先用function,function中使用$1指代传入的第一个参数
#-------------------------------------------------------------#
# screen中如何创建日志?
按 CTRL + A + H 创建日志文件。需要注意,A 和 H 都是大写。
#-------------------------------------------------------------#
# 添加用户sd,并且添加家目录
useradd sd -m

# 把sd添加到sudo组
usermod -aG sudo sd

# 删除用户,包括home目录,强制删除
userdel -rf sd
#-------------------------------------------------------------#
# parallel 安装与使用
sudo apt install parallel 
cat <file-name> | parallel -j 200 curl -L -o /dev/null {} -x 127.0.0.1:8080 -k -s
#-------------------------------------------------------------#
# grep打印文件差集
# 打印出FB独有的行
grep -Fxvf fA fB
# -F 将模式当作是字符串,而不是正则表达式
# -x 只显示匹配的整行,相当于给正则加^...$
# -v 反向选择 如果是grep -Fxf fA fB则是显示既出现在fA中也出现在fB中行,即两者交集
# -f 将fA的每一行当作正则
#-------------------------------------------------------------#
# comm 专业比较文件(需要文件排好序)
# -1 不输出fA独有的
# -2 不输出fB独有的
# -3 不输出公共的
# 输出FB独有的行
comm -13 fA fB
# 输出FA独有的行
comm -23 fA fB
# 输出公共的行
comm -12 fA fB
#-------------------------------------------------------------#
# find技巧:找到并删除空文件夹
find /dir -type d -empty -delete
#-------------------------------------------------------------#
# 适用于 cmder上面执行,要有wsl
# idea 全家桶白嫖
wsl -d ubuntu bash -c "find /mnt/c/Users/*/AppData/Roaming/JetBrains -name *.evaluation.key |xargs -i mv {} {}.bak.$(date +%s)"
wsl -d ubuntu bash -c "find /mnt/c/Users/*/AppData/Roaming/JetBrains -name other.xml |xargs -i mv {} {}.bak.$(date +%s)"
reg delete hkcu\SOFTWARE\JavaSoft\Prefs\jetbrains /f
#-------------------------------------------------------------#
# hex decode
echo 54657374696e672031203220330 | xxd -r -p
#-------------------------------------------------------------#

一键apt install常用工具

1
sudo apt-get install -y git wget curl proxychains4 libpcap-dev build-essential ripgrep parallel dnsutils make jq xsel parallel

useful alias

1
2
alias pbcopy='xsel --clipboard --input'
alias pbpaste='xsel --clipboard --output'

wsl alias

~/.automan/automan.bashrc

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# add golang env
export PATH=$PATH:/usr/local/go/bin:$HOME/go/bin

# custome alias
alias yupdate_custom_alias='source ~/.automan/automan.bashrc'
alias rm='rm -i '
# cd ~
#service cron start >/dev/null

alias pc='proxychains4 -q'
alias wakepy='conda activate d'
alias sleppy='conda deactivate'

alias startwiki="ruby /mnt/d/typote/note/tiddly/tw5.rb &"
alias pbcopy="tee <&0  | clip.exe"
alias pbcopyz="tee <&0 | sed -rz 's/\s*$//g' | clip.exe"  # 去除trailing的换行或空白符等
# refer: https://www.techtronic.us/pbcopy-pbpaste-for-wsl/
alias pbpaste="powershell.exe Get-Clipboard | sed 's/\r$//' | sed -z '$ s/\n$//'"

alias fmtjs="~/.automan/fmtjs.sh"

# gf的补全提示
complete -W "\$(gf -list)" gf

# bash 自动补全
bind '"\e[A": history-search-backward'
bind '"\e[B": history-search-forward'

git

1
git add -A 和 git add -a 是不同的。前者会包含新增的文件,后者只包含修改和删除的文件。git add -A 等价于 git add . 加上 git add -u 。 

Debug

缓冲区问题

程序打印出问题的时候很多时候是输出的缓冲区导致的

C 语言输出无缓存

1
2
3
setbuf(stdout, NULL); 
//或者
fflush(stdout); 

python 输出无缓存

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
方法一:执行脚本时添加参数 python -u script.py

方法二:python>3.3 可以使用 print(123,flush=True)

方法三:
全局改写 print(https://stackoverflow.com/a/35467658/15103280)
import functools
print = functools.partial(print, flush=True)

方法四:
重新打开 stdout (https://stackoverflow.com/a/9462099/15103280)
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

方法五:
设置环境变量。PYTHONUNBUFFERED 设为非空时等价于-u 参数
Linux or OSX:
$ export PYTHONUNBUFFERED=TRUE

Windows:
C:\SET PYTHONUNBUFFERED=TRUE

方法六:
在代码中手动设置环境变量
os.environ['PYTHONUNBUFFERED']="TRUE"

golang 输出无缓存

1
2
bob@pb:/mnt/d/gitrepo/automan/hissl$ head yy_22_open_port.txt | GOCACHE=off  go run hissh.go 2>&1 |tee -a res
build cache is disabled by GOCACHE=off, but required as of Go 1.12

工具安装

1
2
3
4
5
6
7
8
go install -v github.com/projectdiscovery/nuclei/v2/cmd/nuclei@latest
go install -v github.com/projectdiscovery/naabu/v2/cmd/naabu@latest
go install -v github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest
go install -v github.com/projectdiscovery/httpx/cmd/httpx@latest
go install -v github.com/projectdiscovery/katana/cmd/katana@latest
go install -v github.com/tomnomnom/gf@latest
go install -v github.com/tomnomnom/anew@latest
go install -v github.com/projectdiscovery/dnsx/cmd/dnsx@latest

pd命令

1
cat domain | httpx -x ALL -follow-redirects -threads 100 -cdn -content-type -cname -status-code -vhost -web-server -websocket -no-color -title --tls-probe -srd httpx.out -sr | tee -a httpx.log

可复用/有用的代码段

python

request库使用

1
2
import urllib3
urllib3.disable_warnings()

获取当前时间

1
2
import time
print(time.strftime("%Y-%m-%d %H:%M:%S"))

python3错误和捕获

1
2
3
4
try:
	dosomething()
except Exception as e:
    pass

bash

批量格式化srgs收集到的JavaScript文件:fmtjs.sh

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
cd ~

home="/home/bob"
statfile=$home"/.automan/dbs/burpjs/jsbeautify.done"
sdb="/mnt/d/record/burphttp/sitedb/"

printf "formatted js number:\t"
wc -l $statfile | cut -d " " -f 1

printf "new js number:\t"
echo $(find $sdb -type f -name "*.js" | sort -u | comm -23 - $statfile|wc -l)

printf "formating..."
newfiles=$(find $sdb -type f -name "*.js" | sort -u | comm -23 - $statfile | tr '\n' '\0' | parallel -0 -j8 -I% js-beautify -r % )
echo $(echo "${newfiles}" | grep -P "^beautified"  |wc -l)" done"

# grep -Po "$sdb.*\.js$" 是为了解决js-beautify输出结果可能是相对路径的问题,通过正则提取一次获取绝对路径
echo "${newfiles}" | grep -P "^beautified" | cut -d " " -f 2 | grep -Po "$sdb.*\.js$" | sort -u >> $statfile
sort -uo  $statfile  $statfile
echo "merged statefile"

printf "formatted js number:\t"
wc -l $statfile | cut -d " " -f 1

cd -

Java

输出到剪贴板

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
import java.awt.*;
import java.awt.datatransfer.Clipboard;
import java.awt.datatransfer.StringSelection;
import java.awt.datatransfer.Transferable;

public static void writeTextToClipboard(String s) {
    Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();
    Transferable transferable = new StringSelection(s);
    clipboard.setContents(transferable, null);
}

序列化和反序列化

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
//    https://gist.github.com/andy722/1524968
    public static <T extends Serializable> String serialize(T item) {
        final ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
        final ObjectOutputStream objectOutputStream;
        try {
            objectOutputStream = new ObjectOutputStream(byteArrayOutputStream);
            objectOutputStream.writeObject(item);
            objectOutputStream.close();
            return Base64.getEncoder().encodeToString(byteArrayOutputStream.toByteArray());
        } catch (IOException e) {
            throw new Error(e);
        }
    }

    public static <T extends Serializable> T deserialize(String data) {
        try {
//            byte[] dataBytes = Base64.decode(data, Base64.DEFAULT);
            byte[] dataBytes = Base64.getDecoder().decode(data);
            final ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(dataBytes);
            final ObjectInputStream objectInputStream = new ObjectInputStream(byteArrayInputStream);

            @SuppressWarnings({"unchecked"})
            final T obj = (T) objectInputStream.readObject();

            objectInputStream.close();
            return obj;
        } catch (IOException e) {
            throw new Error(e);
        } catch (ClassNotFoundException e) {
            throw new Error(e);
        }
    }

golang

common

1
2
// convert byte to string
myString := string(myBytes)

golang命令行程序模板

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
package main

import (
	"bufio"
	"flag"
	"fmt"
	"io"
	"os"
	"strings"
	"sync"
)

//https://manytools.org/hacker-tools/ascii-banner/
// Rounded font
const banner = ``

const common_usage = `ishttp: check if a port is http/https in minimal effort
    
    echo qq.com | ishttp`

type Options struct {
	threadNum        int
	InputFile   string
	OutputFile   string
	help bool
    timeout int
}
var options = &Options{}
const symbol_for_os_stdin = "os.Stdin"


func main() {
	initprog()
	doSth()
}


func initprog() {
	flag.IntVar(&options.threadNum,"t",30,"线程数")
    flag.IntVar(&options.timeout,"w",5,"每次请求默认超时时间")
	flag.BoolVar(&options.help,"h",false,"打印帮助")
	flag.StringVar(&options.InputFile,"iL",symbol_for_os_stdin,"输入文件")
	flag.StringVar(&options.OutputFile,"o","os.Stdout","输出文件(当前不支持自定义此选项)")

	flag.Usage = func() {
		fmt.Fprintln(os.Stderr, banner)
		flag.PrintDefaults()
		fmt.Fprintln(os.Stderr, common_usage)
	}
	flag.Parse()
	if (options.InputFile==symbol_for_os_stdin && !hasStdin()) || options.help{
		flag.Usage()
		os.Exit(0)
	}
}


func hasStdin() bool {
	//https://stackoverflow.com/questions/22563616/determine-if-stdin-has-data-with-go
	fi, err := os.Stdin.Stat()
	if err != nil {
		panic(err)
	}
	return fi.Mode()&os.ModeNamedPipe != 0
}


func throwErr(errdesc string, err error) {
	fmt.Fprintln(os.Stderr, strings.Repeat("-", 50))
	fmt.Fprintln(os.Stderr, errdesc)
	if err != nil {
		fmt.Fprintln(os.Stderr, err.Error())
	}
	fmt.Fprintln(os.Stderr, strings.Repeat("-", 50))
}


func doSth() {
	//读取输入
	var inputScanner *bufio.Scanner
	var finput io.Reader
	if options.InputFile==symbol_for_os_stdin {
		finput = os.Stdin
	} else {
		var err error
		finput,err = os.Open(options.InputFile)
		if err!=nil{
			throwErr("打开输入文件失败",err)
		}
	}
	inputScanner = bufio.NewScanner(finput)

	var wg sync.WaitGroup
	var ch = make(chan struct{}, options.threadNum)
	for inputScanner.Scan() {
		entry := inputScanner.Text()
		wg.Add(1)
		ch <- struct{}{} // acquire a token
		go func(single_entry string) {
			defer wg.Done()
			doSthUnit(single_entry)
			<-ch // release the token
		}(entry)
	}
	wg.Wait()
}


func doSthUnit(entry string) {
	fmt.Fprintln(os.Stdout, entry)
}

others

bash命令

工作时间统计

1
pbpaste | grep -Po "\d+(?=min)" | tr '\n' '+'|rev|cut -c2-|rev|bc

工具推荐:APP合规扫描

1
2
3
https://github.com/zhengjim/camille 基于Frida的Android App隐私合规检测辅助工具
https://github.com/bytedance/appshark Appshark is a static taint analysis platform to scan vulnerabilities in an Android app.
https://github.com/allenymt/PrivacySentry 工信部-Android隐私合规整改检测工具,注解+Asm修改字节码的检测方案

nmap top services

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# https://svn.nmap.org/nmap/nmap-services
# https://security.stackexchange.com/questions/78618/is-there-a-nmap-command-to-get-the-top-most-common-ports
# 所有常用端口
bob@pb:~$ cat /usr/share/nmap/nmap-services | grep -Pv "^#" | sort -rk3 | head
http    80/tcp  0.484143        # World Wide Web HTTP
ipp     631/udp 0.450281        # Internet Printing Protocol
snmp    161/udp 0.433467        # Simple Net Mgmt Proto
netbios-ns      137/udp 0.365163        # NETBIOS Name Service
ntp     123/udp 0.330879        # Network Time Protocol
netbios-dgm     138/udp 0.297830        # NETBIOS Datagram Service
ms-sql-m        1434/udp        0.293184        # Microsoft-SQL-Monitor
microsoft-ds    445/udp 0.253118
msrpc   135/udp 0.244452        # Microsoft RPC services
dhcps   67/udp  0.228010        # DHCP/Bootstrap Protocol Server

# 协议类型
# 其中SCTP比较神奇,值得单独研究一下:https://blog.csdn.net/wuxing26jiayou/article/details/79743683
bob@pb:~$ cat /usr/share/nmap/nmap-services | grep -Pv "^#" | sort -rk3 | cut -f2 | cut -d/ -f2 |sort -u
sctp
tcp
udp

# top100 不管协议
$ cat /usr/share/nmap/nmap-services | grep -Pv "^#" | sort -rk3 | cut -f2 | cut -d/ -f1 | head -n100 | tr '\n' ,

# top tcp协议
bob@pb:~$ cat /usr/share/nmap/nmap-services | grep -Pv "^#" | sort -rk3 | cut -f2 | grep tcp | cut -d/ -f1 |wc
   8320    8320   42421
   
# top udp协议
bob@pb:~$ cat /usr/share/nmap/nmap-services | grep -Pv "^#" | sort -rk3 | cut -f2 | grep udp | cut -d/ -f1 |wc
  19022   19022  107028

科学上网参考

https://github.com/emptysuns/Hi_Hysteria

update