Documentation ¶
Index ¶
- Constants
- Variables
- func BuildInstance(ctx context.Context, opts BuildInstanceOptions) error
- func EnsureFile(ctx context.Context, fs FileSystem, path string, content io.Reader) error
- func ExtractInstance(ctx context.Context, inst PackageInstance, dest Destination, ...) error
- func IsCorruptionError(err error) bool
- func ValidateInstallMode(mode InstallMode) error
- type BuildInstanceOptions
- type Deployer
- type Description
- type Destination
- type ExtractFilter
- type File
- func NewTestFile(name string, data string, executable bool) File
- func NewTestSymlink(name string, target string) File
- func NewWinTestFile(name string, data string, attrs WinAttrs) File
- func ScanFileSystem(dir string, root string, exclude ScanFilter) ([]File, error)
- func WrapFile(abs string, root string, fileInfo *os.FileInfo) (File, error)
- type FileInfo
- type FileSystem
- type InstallMode
- type InstanceFile
- type Manifest
- type PackageChunkDef
- type PackageDef
- type PackageInstance
- type ScanFilter
- type VerificationMode
- type VersionFile
- type WinAttrs
Constants ¶
const SiteServiceDir = ".cipd"
SiteServiceDir is a name of the directory inside an installation root reserved for cipd stuff.
Variables ¶
Functions ¶
func BuildInstance ¶
func BuildInstance(ctx context.Context, opts BuildInstanceOptions) error
BuildInstance builds a new package instance.
If build an instance of package named opts.PackageName by archiving input files (passed via opts.Input).
The final binary is written to opts.Output. Some output may be written even if BuildInstance eventually returns an error.
func EnsureFile ¶
EnsureFile creates a file with the given content. It will create full directory path to the file if necessary.
func ExtractInstance ¶
func ExtractInstance(ctx context.Context, inst PackageInstance, dest Destination, exclude ExtractFilter) error
ExtractInstance extracts all files from a package instance into a destination.
If exclude != nil, it will be used to filter the contents of the PackageInstance before extraction.
func IsCorruptionError ¶
IsCorruptionError returns true iff err indicates corruption.
func ValidateInstallMode ¶
func ValidateInstallMode(mode InstallMode) error
ValidateInstallMode returns non nil if install mode is invalid.
Valid modes are: "" (client will pick platform default), "copy" (aka InstallModeCopy), "symlink" (aka InstallModeSymlink).
Types ¶
type BuildInstanceOptions ¶
type BuildInstanceOptions struct { // Input is a list of files to add to the package. Input []File // Output is where to write the package file to. Output io.Writer // PackageName is name of the package being built, e.g. 'infra/tools/cipd'. PackageName string // VersionFile is slash separated path where to drop JSON with version info. VersionFile string // InstallMode defines how to install the package: "copy" or "symlink". InstallMode InstallMode // CompressionLevel defines deflate compression level in range [0-9]. CompressionLevel int }
BuildInstanceOptions defines options for BuildInstance function.
type Deployer ¶
type Deployer interface { // DeployInstance installs an instance of a package into the given subdir of // the root. // // It unpacks the package into <base>/.cipd/pkgs/*, and rearranges // symlinks to point to unpacked files. It tries to make it as "atomic" as // possible. Returns information about the deployed instance. // // Due to a historical bug, if inst contains any files which are intended to // be deployed to `.cipd/*`, they will not be extracted and you'll see // warnings logged. DeployInstance(ctx context.Context, subdir string, inst PackageInstance) (common.Pin, error) // CheckDeployed checks whether a given package is deployed at the given // subdir. // // It returns information about installed version (or error if not installed). CheckDeployed(ctx context.Context, subdir, packageName string) (common.Pin, error) // FindDeployed returns a list of packages deployed to a site root. FindDeployed(ctx context.Context) (out common.PinSliceBySubdir, err error) // RemoveDeployed deletes a package from a subdir given its name. RemoveDeployed(ctx context.Context, subdir, packageName string) error // TempFile returns os.File located in <base>/.cipd/tmp/*. // // The file is open for reading and writing. TempFile(ctx context.Context, prefix string) (*os.File, error) // CleanupTrash attemps to remove stale files. // // May return errors if some files are still locked, this is fine. CleanupTrash(ctx context.Context) error }
Deployer knows how to unzip and place packages into site root directory.
type Description ¶
type Description struct { Subdir string `json:"subdir,omitempty"` PackageName string `json:"package_name,omitempty"` }
Description defines the structure of the description.json file located at .cipd/pkgs/<foo>/description.json.
type Destination ¶
type Destination interface { // Begin initiates a new write transaction. Called before first CreateFile. Begin(ctx context.Context) error // CreateFile opens a writer to extract some package file to. CreateFile(ctx context.Context, name string, executable bool, winAttrs WinAttrs) (io.WriteCloser, error) // CreateSymlink creates a symlink (with absolute or relative target). // // 'name' must be a slash separated path relative to the destination root. CreateSymlink(ctx context.Context, name string, target string) error // End finalizes package extraction (commit or rollback, based on success). End(ctx context.Context, success bool) error }
Destination knows how to create files when extracting a package.
It supports transactional semantic by providing 'Begin' and 'End' methods. No changes should be applied until End(true) is called. A call to End(false) should discard any pending changes. All paths are slash separated.
While between 'Begin' and 'End', 'CreateFile' and 'CreateSymlink' can be called concurrently.
func NewFileSystemDestination ¶
func NewFileSystemDestination(dir string, fs FileSystem) Destination
NewFileSystemDestination returns a destination in the file system (directory) to extract a package to.
Will use a provided FileSystem object to operate on files if given, otherwise use a default one. If FileSystem is provided, dir must be in a subdirectory of the given FileSystem root.
type ExtractFilter ¶
ExtractFilter is predicate used by ExtractInstance to exclude files from extraction and the manifest.json. The function will be given the name of the file inside the instance zip (so, the path relative to the extraction destination).
type File ¶
type File interface { // Name returns slash separated file path relative to a package root // // For example "dir/dir/file". Name() string // Size returns size of the file. 0 for symlinks. Size() uint64 // Executable returns true if the file is executable. // // Only used for Linux\Mac archives. false for symlinks. Executable() bool // Symlink returns true if the file is a symlink. Symlink() bool // SymlinkTarget return a path the symlink is pointing to. SymlinkTarget() (string, error) // WinAttrs returns the windows attributes, if any. WinAttrs() WinAttrs // Open opens the regular file for reading. // // Returns error for symlink files. Open() (io.ReadCloser, error) }
File defines a single file to be added or extracted from a package.
All paths are slash separated (including symlink targets).
func NewTestFile ¶
NewTestFile returns File implementation (Symlink == false) backed by a fake in-memory data. It is useful in unit tests.
func NewTestSymlink ¶
NewTestSymlink returns File implementation (Symlink == true) backed by a fake in-memory data. It is useful in unit tests.
func NewWinTestFile ¶
NewWinTestFile returns a File implementation (Symlink == false, Executable == false) backed by a fake in-memory data with windows attributes. It is useful in unit tests.
func ScanFileSystem ¶
func ScanFileSystem(dir string, root string, exclude ScanFilter) ([]File, error)
ScanFileSystem returns all files in some file system directory in an alphabetical order. It returns only files, skipping directory entries (i.e. empty directories are completely invisible).
ScanFileSystem follows symbolic links which have a target in <root>/<SiteServiceDir> (i.e. the .cipd folder). Other symlinks will will be preserved as symlinks (see Symlink() method of File interface). Symlinks with absolute targets inside of the root will be converted to relative targets inside of the root.
It scans "dir" path, returning File objects that have paths relative to "root". It skips files and directories for which 'exclude(absolute path)' returns true. It also will always skip <root>/<SiteServiceDir>.
func WrapFile ¶
WrapFile constructs File object for some file in the file system, specified by its native absolute path 'abs' (subpath of 'root', also specified as a native absolute path). Returned File object has path relative to 'root'. If fileInfo is given, it will be used to grab file mode and size, otherwise os.Lstat will be used to get it. Recognizes symlinks.
type FileInfo ¶
type FileInfo struct { // Name is slash separated file path relative to a package root. Name string `json:"name"` // Size is a size of the file. 0 for symlinks. Size uint64 `json:"size"` // Executable is true if the file is executable. // // Only used for Linux\Mac archives. False for symlinks. Executable bool `json:"executable,omitempty"` // WinAttrs is a string representation of extra windows file attributes. // // Only used for Win archives. WinAttrs string `json:"win_attrs,omitempty"` // Symlink is a path the symlink points to or "" if the file is not a symlink. Symlink string `json:"symlink,omitempty"` }
FileInfo is JSON-ish struct with info extracted from File interface.
type FileSystem ¶
type FileSystem interface { // Root returns absolute path to a directory FileSystem operates in. // // All FS actions are restricted to this directory. Root() string // CwdRelToAbs converts a path relative to cwd to an absolute one. // // If also verifies the path is under the root path of the FileSystem object. // If passed path is already absolute, just checks that it's under the root. CwdRelToAbs(path string) (string, error) // RootRelToAbs converts a path relative to Root() to an absolute one. // // It verifies the path is under the root path of the FileSystem object. // If passed path is already absolute, just checks that it's under the root. RootRelToAbs(path string) (string, error) // EnsureDirectory creates a directory at given native path. // // Does nothing it the path already exists. It takes an absolute path or // a path relative to the current working directory and always returns // absolute path. EnsureDirectory(ctx context.Context, path string) (string, error) // EnsureSymlink creates a symlink pointing to a target. // // It will create full directory path to the symlink if necessary. EnsureSymlink(ctx context.Context, path string, target string) error // EnsureFile creates a file and calls the function to write file content. // // It will create full directory path to the file if necessary. EnsureFile(ctx context.Context, path string, write func(*os.File) error) error // EnsureFileGone removes a file, logging the errors (if any). // // Missing file is not an error. EnsureFileGone(ctx context.Context, path string) error // EnsureDirectoryGone recursively removes a directory. EnsureDirectoryGone(ctx context.Context, path string) error // Renames oldpath to newpath. // // If newpath already exists (be it a file or a directory), removes it first. // If oldpath is a symlink, it's moved as is (e.g. as a symlink). Replace(ctx context.Context, oldpath, newpath string) error // CleanupTrash attempts to remove all files that ended up in the trash. // // This is best effort operation. CleanupTrash(ctx context.Context) error }
FileSystem abstracts operations that touch single file system subpath.
All functions operate in terms of native file paths. It exists mostly to hide differences between file system semantic on Windows and Linux\Mac.
func NewFileSystem ¶
func NewFileSystem(root, trash string) FileSystem
NewFileSystem returns default FileSystem implementation.
It operates with files under a given path. All methods accept absolute paths or paths relative to current working directory. FileSystem will ensure they are under 'root' directory.
It can also accept a path to a directory to put "trash" into: files that can't be removed because there are some processes keeping lock on them. This is useful on Windows when replacing running executables. The trash directory must be on the same disk as the root directory.
It 'trash' is empty string, the trash directory will be created under 'root'.
type InstallMode ¶
type InstallMode string
InstallMode defines how to install a package.
const ( // InstallModeSymlink is default (for backward compatibility). // // In this mode all files are extracted to .cipd/*/... and then symlinked to // the site root directory. Version switch happens atomically. InstallModeSymlink InstallMode = "symlink" // InstallModeCopy is used when files should be directly copied. // // This mode is always used on Windows (and can be optionally) used on // other OSes. If installation is aborted midway, the package may end up // in inconsistent state. InstallModeCopy InstallMode = "copy" )
func (*InstallMode) Set ¶
func (m *InstallMode) Set(value string) error
Set is called by 'flag' package when parsing command line options.
func (InstallMode) String ¶
func (m InstallMode) String() string
String is needed to conform to flag.Value interface.
type InstanceFile ¶
type InstanceFile interface { io.ReadSeeker // Close is a bit non-standard, and can be used to indicate to the storage // (filesystem and/or cache) layer that this instance is actaully bad. The // storage layer can then evict/revoke, etc. the bad file. Close(ctx context.Context, corrupt bool) error }
InstanceFile is an underlying data file for a PackageInstance.
type Manifest ¶
type Manifest struct { FormatVersion string `json:"format_version"` PackageName string `json:"package_name"` VersionFile string `json:"version_file,omitempty"` // where to put JSON with info about deployed package InstallMode InstallMode `json:"install_mode,omitempty"` // how to install: "copy" or "symlink" Files []FileInfo `json:"files,omitempty"` // present only in deployed manifest }
Manifest defines structure of manifest.json file.
type PackageChunkDef ¶
type PackageChunkDef struct { // Dir is a directory to add to the package (recursively). Dir string // File is a single file to add to the package. File string // VersionFile defines where to drop JSON file with package version. VersionFile string `yaml:"version_file"` // Exclude is a list of regexp patterns to exclude when scanning a directory. Exclude []string }
PackageChunkDef represents one entry in 'data' section of package definition.
It is either a single file, or a recursively scanned directory (with optional list of regexps for files to skip).
type PackageDef ¶
type PackageDef struct { // Package defines a name of the package. Package string // Root defines where to search for files. It may either be an absolute path, // or it may be a path relative to the package file itself. If omitted, it // defaults to "." (i.e., the same directory as the package file) Root string // InstallMode defines how to deploy the package file: "copy" or "symlink". InstallMode InstallMode `yaml:"install_mode"` // Data describes what is deployed with the package. Data []PackageChunkDef }
PackageDef defines how exactly to build a package.
It specified what files to put into it, how to name them, how to name the package itself, etc. It is loaded from *.yaml file.
func LoadPackageDef ¶
LoadPackageDef loads package definition from a YAML source code.
It substitutes %{...} strings in the definition with corresponding values from 'vars' map.
func (*PackageDef) FindFiles ¶
func (def *PackageDef) FindFiles(cwd string) ([]File, error)
FindFiles scans files system and returns files to be added to the package.
It uses a path to package definition file directory ('cwd' argument) to find a root of the package.
func (*PackageDef) VersionFile ¶
func (def *PackageDef) VersionFile() string
VersionFile defines where to drop JSON file with package version.
type PackageInstance ¶
type PackageInstance interface { // Pin identifies package name and concreted instance ID of this package file. Pin() common.Pin // Files returns a list of files to deploy with the package. Files() []File // DataReader returns reader that reads raw package data. DataReader() io.ReadSeeker }
PackageInstance represents a binary CIPD package file (with manifest inside).
func OpenInstance ¶
func OpenInstance(ctx context.Context, r InstanceFile, instanceID string, v VerificationMode) (PackageInstance, error)
OpenInstance prepares the package for extraction.
If instanceID is an empty string, OpenInstance will calculate the hash of the package and use it as InstanceID (regardless of verification mode).
If instanceID is not empty and verification mode is VerifyHash, OpenInstance will check that package data matches the given instanceID. It skips this check if verification mode is SkipHashVerification.
func OpenInstanceFile ¶
func OpenInstanceFile(ctx context.Context, path string, instanceID string, v VerificationMode) (inst PackageInstance, closer func() error, err error)
OpenInstanceFile opens a package instance file on disk.
The caller of this function must call closer() if err != nil to close the underlying file.
type ScanFilter ¶
ScanFilter is predicate used by ScanFileSystem to decide what files to skip.
type VerificationMode ¶
type VerificationMode int
VerificationMode is defines whether to verify hash or not.
const ( // VerifyHash instructs OpenPackage to calculate hash of the package and // compare it to the given instanceID. VerifyHash VerificationMode = 0 // SkipVerification instructs OpenPackage to skip the hash verification and // trust that the given instanceID matches the package. SkipHashVerification VerificationMode = 1 )
type VersionFile ¶
type VersionFile struct { PackageName string `json:"package_name"` InstanceID string `json:"instance_id"` }
VersionFile describes JSON file with package version information that's deployed to a path specified in 'version_file' attribute of the manifest.
type WinAttrs ¶
type WinAttrs uint32
WinAttrs represents the extra file attributes for windows.
const ( WinAttrHidden WinAttrs = 0x2 WinAttrSystem WinAttrs = 0x4 WinAttrsAll WinAttrs = WinAttrHidden | WinAttrSystem )
These are the valid WinAttrs values. They may be OR'd together to form a mask. These match the windows GetFileAttributes values.