mallory

package module
v0.0.0-...-64c18fd Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 17, 2016 License: MIT Imports: 30 Imported by: 0

README

mallory

Gitter

Yet another http proxy written in golang, including direct, GAE, SOCKS5, and SSH fetchers.

Status Build Status

  • Support direct fetcher that spawn links from the running machine.
  • Support GAE fetcher, only support http and https with port 443. In this mode we need to deploy fake certificates to forward https requests...
  • Support SOCKS5 proxy fetcher, aka SOCKS5 to HTTP proxy translator.
  • Support SSH fetcher, aka HTTP proxy via SSH tunnel.
  • Simple PAC file server.

Installation

go get gopkg.in/ratson/mallory.v0/cmd/mallory

Engines

Direct
# This is the default mode, that spawns connections from the running machine.
# Now we have the HTTP proxy on port 1315
mallory
2014/04/12 01:56:33 Listen and serve HTTP proxy on 127.0.0.1:1315
2014/04/12 01:56:33 	Engine: direct
GAE
# This engine spawns connections from the remote Google Application Engine.

# copy config to the default work dir
mkdir ~/.mallory && cp cfg/* ~/.mallory

# before start the proxy server, we'd better upload the GAE remote application
# for details see https://appengine.google.com
cd mallory/

# put your own app id into app.yaml
vim gae_server/app.yaml

# deploy it with go_appengine, https://developers.google.com/appengine/downloads
goapp deploy gae_server/

# this mode need to use the fake CA, default include crt and key
mallory -engine=gae -remote=https://your-app-id.appspot.com

# or generate the private key and sign the Root CA by yourself
openssl genrsa -out key 2048
openssl req -new -x509 -days 365 -key key -out crt
SOCKS
# This engine spawns connections from the remote SOCKS proxy server,
# a simple way to translate SOCKS proxy to HTTP proxy.
# Assume we have a SOCKS5 proxy server at localhost and listen on port 1314
# Now we have the HTTP proxy on port 1315
mallory -engine=socks -remote=socks5://localhost:1314
2014/06/19 16:39:05 Starting...
2014/06/19 16:39:05 Listen and serve HTTP proxy on 127.0.0.1:1315
2014/06/19 16:39:05 	Engine: socks
2014/06/19 16:39:05 	Remote SOCKS proxy server: socks5://localhost:1314
SSH
# This engine spawns connections from the remote SSH server, similar to the ssh -D command.
# The difference between them is that:
#   ssh -D  ==> SOCKS proxy
#   mallory ==> HTTP proxy
# Assume we have a ssh server on linode:22
# Now we have the HTTP proxy on port 1315
mallory -engine=ssh -remote=ssh://linode:22
2014/06/19 16:45:12 Starting...
2014/06/19 16:45:13 Listen and serve HTTP proxy on 127.0.0.1:1315
2014/06/19 16:45:13 	Engine: ssh
2014/06/19 16:45:13 	Remote SSH server: ssh://linode:22

# Add username, password and custom port 1234
mallory -engine=ssh -remote=ssh://user:password@linode:1234

TODO

  • Add test
  • ....

References

Documentation

Overview

Package mallory implements a simple http proxy support direct and GAE remote fetcher

Index

Constants

View Source 
const (
	CO_RED    = "\033[0;31;49m"
	CO_GREEN  = "\033[0;32;49m"
	CO_YELLOW = "\033[0;33;49m"
	CO_BLUE   = "\033[0;34;49m"
	CO_RESET  = "\033[0m"
)

Variables

This section is empty.

Functions

func BeautifyDuration 

func BeautifyDuration(d time.Duration) string

Duration to e.g. 432ms or 12s, human readable translation

func BeautifySize 

func BeautifySize(s int64) string

func CopyHeader 

func CopyHeader(w http.ResponseWriter, r *http.Response)

copy and overwrite headers from r to w

func Ioctl 

func Ioctl(fd, request, argp uintptr) syscall.Errno

POSIX ioctl syscall From https://github.com/mreiferson/go-simplelog/blob/master/simplelog.go

func IsExist 

func IsExist(path string) bool

Test a file is exist or not

func Isatty 

func Isatty(f *os.File) bool

Test is a termnial or not From https://github.com/mreiferson/go-simplelog/blob/master/simplelog.go

func StatusText 

func StatusText(c int) string

Return http status text looks like "200 OK"

Types

type CertPool 

type CertPool struct {
	// where to store certificates
	Dir string
	// CA to sign new certs
	CA *tls.Certificate
	// contains filtered or unexported fields
}

certificates pool, able to get and create certificates thread safely

func NewCertPool 

func NewCertPool(dir string, ca *tls.Certificate) (self *CertPool)

Create pool. dir is the path where is save and load certificates ca is the root CA to create and sign all new certificates

func (*CertPool) Get 

func (self *CertPool) Get(host string) (cert *tls.Certificate, err error)

type Engine 

type Engine interface {
	// normal http methods except CONNECT
	// all operations in this function should be thread safe
	Serve(*Session)
	// handle CONNECT method, a secure tunnel
	// all operations in this function should be thread safe
	// Tunneling TCP based protocols through Web proxy servers
	//  - http://www.web-cache.com/Writings/Internet-Drafts/draft-luotonen-web-proxy-tunneling-01.txt
	Connect(*Session)
}

The method to fetch data from remote server or connect to another proxy server or something...

type EngineDirect 

type EngineDirect struct {
	Tr *http.Transport
}

Direct fetcher from the host of proxy

func CreateEngineDirect 

func CreateEngineDirect(e *Env) (*EngineDirect, error)

Create and initialize

func (*EngineDirect) Connect 

func (self *EngineDirect) Connect(s *Session)

Data flow:

  1. Receive CONNECT request from the client
  2. Dial the remote server(the one client want to conenct)
  3. Send 200 OK to client if the connection is established
  4. Exchange data between client and server

func (*EngineDirect) Serve 

func (self *EngineDirect) Serve(s *Session)

Data flow:

  1. Receive request R1 from client
  2. Re-post request R1 to remote server(the one client want to connect)
  3. Receive response P1 from remote server
  4. Send response P1 to client

type EngineGAE 

type EngineGAE struct {
	// Global config
	Env *Env
	// work space for this engine
	Work string
	// place store certificates
	CertDir string
	// Loaded certificate, contains the root certificate and private key
	RootCA *tls.Certificate
	// Pool of auto generated fake certificates signed by RootCert
	Certs *CertPool
}

all write on this should be sync between threads

func CreateEngineGAE 

func CreateEngineGAE(e *Env) (self *EngineGAE, err error)

Create and initialize

func (*EngineGAE) Connect 

func (self *EngineGAE) Connect(s *Session)
Impossible to connect gae and handle it as a normal TCP connection?
GAE only provide http handlers? At least I don't know how to handle to TCP connection on GAE server.
NOTE: GAE socket service can only be available for billing users. So free users is unable to use the
long term connection. And do what we did in EngineDirect.
So we can only use urlfetch.Client.Transport.RoundTrip to do http or https method.
Generally, the CONNECT method can be used for any purpose for the advantage of TCP connection.
The proxy doesn't need to know what the real underlying protocol or what it is, just need to copy
data from client to server, and copy the response from the server to client without any interpret.
Now what we can do and had been done by some GAE proxies is that, extract the underlying protocol!!!
GAE can only handle limited protocols with urlfetch module, such as http and https.
Use Hijacker to get the underlying connection

Data flow:

  1. Detect host and port
  2. Hijack the client connection
  3. Dial self
  4. Return 200 OK if is successfully
  5. Get cached or create new signed certificate
  6. Wrap client connection with TLS and make handshake
  7. Receive http request
  8. Write request as a proxy request to self, HTTP handler
  9. Copy response to client...

func (*EngineGAE) Serve 

func (self *EngineGAE) Serve(s *Session)

Data flow:

  1. Receive client request R1
  2. Write R1 as the body of a new request R2
  3. Post request R2 to remote GAE
  4. Receive response P1 from GAE
  5. Read remote server(which the client want to connect with) resonse P2 from the body of P1
  6. Send P2 as the response to client

type EngineSOCKS 

type EngineSOCKS struct {
	Env *Env
	Dir *EngineDirect
}

translate SOCKS proxy to HTTP proxy

func CreateEngineSOCKS 

func CreateEngineSOCKS(e *Env) (self *EngineSOCKS, err error)

Create and initialize

func (*EngineSOCKS) Connect 

func (self *EngineSOCKS) Connect(s *Session)

func (*EngineSOCKS) Serve 

func (self *EngineSOCKS) Serve(s *Session)

type EngineSSH 

type EngineSSH struct {
	Env *Env
	URL *url.URL
	Cli *ssh.Client
	Cfg *ssh.ClientConfig
	Dir *EngineDirect
	// contains filtered or unexported fields
}

func CreateEngineSSH 

func CreateEngineSSH(e *Env) (self *EngineSSH, err error)

Create and initialize

func (*EngineSSH) Connect 

func (self *EngineSSH) Connect(s *Session)

func (*EngineSSH) Serve 

func (self *EngineSSH) Serve(s *Session)

type Env 

type Env struct {
	// work space, default is $HOME/.mallory
	Work string
	// local addr to listen and serve, default is 127.0.0.1:1315
	Addr string
	// remote engine to be used, "direct"(default), "gae", "socks", or "ssh"
	Engine string
	// 1) GAE application URL, only valid when the engine is "gae"
	//    e.g. https://kill-me-baby.appspot.com
	// 2) URL of socks proxy, only valid when the engine is "socks"
	//    e.g. socks5://localhost:1314
	// 3) URL of ssh server, only valid when the engine is "ssh"
	//    e.g. ssh://user:[email protected]:1122
	Remote string
	// > http://www.akadia.com/services/ssh_test_certificate.html
	// > http://mitmproxy.org/doc/ssl.html
	// RSA private key file and self-signed root certificate file
	// Can be generated by OpenSSL:
	// - RSA private key file, without input any extra info
	//      openssl genrsa -out key 2048
	// - Self-signed root certificate file, need input some X.509 attributes
	//   such as Country Name, Comman Name etc.
	//      openssl req -new -x509 -days 365 -key mollory.key -out crt
	Key  string // key, only valid when engine is "gae", default is work/key
	Cert string // crt, only valid when engine is "gae", default is work/crt
	// pac file path, default is work/pac
	PAC string
	// terminal helper, test the default logger(os.Stderr) is terminal or not
	Istty bool
}

Provide global config for mallory

func (*Env) Parse 

func (self *Env) Parse() error

Prepare flags and env

type Server 

type Server struct {
	// Global config
	Env *Env
	// different fetch engine can be adapted to the server
	Engine Engine
	// services provided
	Services map[string]Service
	// alive connections
	CountAlive int64
	// contains filtered or unexported fields
}

The main proxy http handler

func CreateServer 

func CreateServer(e *Env) (self *Server, err error)

Create and intialize

func (*Server) NewID 

func (self *Server) NewID() int64

Return a new unique ID, thread safe

func (*Server) Reg 

func (self *Server) Reg(s Service)

Register a service to the server, later service will overwrite the previous one if both of them have the same service path

func (*Server) ServeHTTP 

func (self *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)

HTTP proxy accepts requests with following two types:

  • CONNECT Generally, this method is used when the client want to connect server with HTTPS. In fact, the client can do anything he want in this CONNECT way... The request is something like: CONNECT www.google.com:443 HTTP/1.1 Only has the host and port information, and the proxy should not do anything with the underlying data. What the proxy can do is just exchange data between client and server. After accepting this, the proxy should response HTTP/1.1 200 OK to the client if the connection to the remote server is established. Then client and server start to exchange data...

  • non-CONNECT, such as GET, POST, ... In this case, the proxy should redo the method to the remote server. All of these methods should have the absolute URL that contains the host information. A GET request looks like: GET weibo.com/justmao945/.... HTTP/1.1 which is different from the normal http request: GET /justmao945/... HTTP/1.1 Because we can be sure that all of them are http request, we can only redo the request to the remote server and copy the reponse to client.

type Service 

type Service interface {
	// serve a http request
	Serve(*Session)
	// the path served by this service, e.g. /pac
	Path() string
}

Extra services provied by server

type ServiceFavicon 

type ServiceFavicon struct{}

Provide a simple service for favicon

func (*ServiceFavicon) Path 

func (*ServiceFavicon) Path() string

return "/favicon.ico"

func (*ServiceFavicon) Serve 

func (*ServiceFavicon) Serve(s *Session)

main handler

type ServicePAC 

type ServicePAC struct {
	// Global config
	Env *Env
	Url *url.URL
}

Provide a simple service for PAC file

func CreateServicePAC 

func CreateServicePAC(e *Env) (self *ServicePAC, err error)

create and init

func (*ServicePAC) Path 

func (self *ServicePAC) Path() string

return "/pac"

func (*ServicePAC) Serve 

func (self *ServicePAC) Serve(s *Session)

main handler, read file and response, please don't use cache

type Session 

type Session struct {
	// Global server
	Server *Server
	// the unique ID start from 1
	ID int64
	// Copy from the http handler
	ResponseWriter http.ResponseWriter
	Request        *http.Request
}

A session is a proxy request

func NewSession 

func NewSession(s *Server, w http.ResponseWriter, r *http.Request) *Session

func (*Session) Error 

func (self *Session) Error(format string, args ...interface{})

func (*Session) Info 

func (self *Session) Info(format string, args ...interface{})

func (*Session) Warn 

func (self *Session) Warn(format string, args ...interface{})

Source Files

View all Source files

Directories

Path Synopsis
cmd
mallory
gae_server
fetcher
Package fetcher implements a simple bridge for HTTP request and response
 Package fetcher implements a simple bridge for HTTP request and response
Package proxy provides support for a variety of protocols to proxy network data.
 Package proxy provides support for a variety of protocols to proxy network data.
ssh
Package ssh implements an SSH client and server.
 Package ssh implements an SSH client and server.
agent
Package agent implements a client to an ssh-agent daemon.
 Package agent implements a client to an ssh-agent daemon.
terminal
Package terminal provides support functions for dealing with terminals, as commonly found on UNIX systems.
 Package terminal provides support functions for dealing with terminals, as commonly found on UNIX systems.
test
This package contains integration tests for the gopkg.in/ratson/mallory.v0/ssh package.
 This package contains integration tests for the gopkg.in/ratson/mallory.v0/ssh package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.