Sash - the Swiss Army Shell

Documentation date: 24th December 2004

Sash is a unix shell and associated scripting language with unusually good support for networking, string handling and images. It also has a large array of miscellaneous functions for the digital equivalent of getting stones out of horses' hooves.

The layout of this document is to rush through enough to get you started writing scripts, then fill in detail for reference. In this first version only the first rush and minimal reference tables are included.

Basics

Any expression which isn't part of something else is written to standard output, so everyone's first sash script is:
#!/usr/bin/env sash

"Hello world\n"			# everything from the hash to end of line is a comment
The first line identifies the script as needing the sash interpreter to run it, and avoids hard-coding the location of the interpreter by using the unix 'env' command to find it on your path.

Basic arithmetic and logical operations (+-*/%&|~) work as in other languages, and there are no syntactic boundaries between statements so you can say:

#!/usr/bin/env sash

2 + 3 * 5 "\n"
to output 17 with a newline after it.

Simple variables of integer or string type can be used much as in other languages, and always have names that start with '$'. They are generally strongly typed when created by budding off from their parent variables: $integers and $strings using the 'new' attribute (explained more later). For example:

#!/usr/bin/env sash

$celsius = $integers.new
$fahrenheight = $integers.new
$result = $strings.new

$celsius = 20
$fahrenheight = ($celsius * 9) / 5 + 32
$result = "%d degrees Celsius is %d degrees Fahrenheight\n", $celsius, $fahrenheight

$result

Numbers can be compared in the usual ways (==, !=, <, >, <=, >=) and strings can be simply compared for equality or inequality (the other string comparison operators are more exotic). Comparisons can be used in if and while statements, which are represented by '?' and '??' respectively as sash has no textual keywords to distract from the programmer's own words. Both if and while always control blocks of code enclosed in curly brackets {}, and an if block can be followed by an else block, where else is the negated if !?. For example:

#!/usr/bin/env sash

$count = $integers.new

$count = 1

"I can count to 6\n"

?? $count <= 6
{
	"Count is %d, ", $count
	? $count & 1
	{
		"%d is odd\n", $count
	}
	!?
	{
		"%d is even\n", $count
	}
	
	$count = $count + 1
}

All types of data are supported by a range of 'attributes' which provide processing of the data or access to related data. There are currently about 150 of these and more will be added as things which are frequently needed and/or easier to implement in the interpreter than a script are identified.

Attributes are added to the data item with a dot, for example if $name is a string the $name.upper is the same string in upper case, $name.lower is the lower case version, and $name.mixed is lower case apart from the first letter of each word. Attributes can be combined with left to right execution, for example $system.now is the current time in seconds since the start of (sash) time (useful for calculations but horrible to show to people) and date_string is the string attribute of a number that makes it a user-readable date so $system.now.date_time is the presentable form of the current time.

#!/usr/bin/env sash

$roll = $integers.new

$roll = 6
"D6 result is %d\n", $roll.die		# get a random number in range set by the variable
"D20 result is %d\n", 20.die		# constants can also have attributes

Running this script repeatedly gives the same 'random' numbers because the pseudo-random generator is always set the same way by default, which is useful for re-running a turn without random variation but bad when the results need to really vary. Attributes can also be used on the left hand side of assignments as in this example:
#!/usr/bin/env sash

$roll = $integers.new

$system.random = $system.now		# start pseudo-random sequence from clock time

$roll = 6
"D6 result is %d\n", $roll.die		# get a random number in range set by the variable
"D20 result is %d\n", 20.die		# constants can also have attributes

Results now vary unless the script is run twice in the same second (which can be fixed for most practical cases by using $system.utime, the microsecond time-of-day clock).

Networking

While most shell script languages are basically about file handling, sash is basically about network handling. A combination of the language, the interpreter and some scripts (also in sash) combine to make implementing network services almost as trivial as implementing subroutines in other languages. In fact sash doesn't have subroutines as large sash programs tend to split naturally into multiple net services rather than multiple blocks within the same script.

Sash does also include more sophisticated file handling than other script languages, just as it contains better string support and image processing, but without the networking it would remain just an incremental advance. It would also need some way to read from the keyboard, rather than expecting input to come from the network as now.

There are two types of network variable: client and server. A client variable is used to make a connection to a chosen other computer and TCP/IP port, expecting the other end to be a server already waiting for connections (for example, a web server). A server variable does not immediately connect and does not choose the other end: it just starts waiting for something (for example a web browser) to connect to it. Sash uses elaborate support such as named services when connecting to other sash scripts, but can also use legacy methods such as numbered TCP ports when connecting with non-sash programs.

Here is a script to fetch a web page, using a client variable to connect to a web server and talk http protocol directly to it.

#!/usr/bin/env sash

$webserver = $clients.new		# get a variable of type client
$result = $strings.new

$webserver.target = "www.pbm.com:80"	# aim it at port 80 on the remote web server
$webserver.packet = "GET /index.html HTTP/1.0\n\n"	# send request for top level web page
$result = $webserver.content		# read entire result

"Web page is %s\n", $result
Note contrast between 'packet' attribute which reads or writes available data while keeping the connecton open, and the 'content' attribute which when writing writes all the data and then closes the connection and when reading continues reading until the other end of the connection closes. It's generally vital to get this distinction right as using 'content' too early will break the connection before all data has been exchanged, and using 'packet' for the last transfer can leave the other end hanging around expecting something else to happen.

Here is a script to implement a simple web server, which listens on a spare port (80 is likely to be in use by a real web server) and replies to any browser request with a place-holder web page.

#!/usr/bin/env sash
# while this script is running on your machine you should be able to connect from a web browser
# with URL http://localhost:8080/anything

$listener = $servers.new
$client = $clients.new
$request = $strings.new

$listener.update =			# set a block of code to run when something connects
{
	$client = $listener.accept	# this variable holds specific client connection
	$request = $client.packet	# see what the client wants, but assume it's a webpage for now
	$client.content = "HTTP/1.0 200 ok\nContent-type: text/html\n\n<h1>Simple Sash Web Server</h1>\n"
	# reply with http headers and a web page, then close the connection
}

$listener.port = 8080			# choose an unused port
:: 0					# this means wait forever
Note use of the 'update' attribute to define the block of code to run when something happens to the listening server variable. This handler uses a client type variable to form a specific connection with the requesting web browser. It turns out that client variables of this sort behave in exactly the same way as client variables described above for initiating connections, so there's no need for the interpreter or programmer to treat the two cases differently.

It would be possible to expand this skeleton script to be a complete server (and this is substantially what's done to make non-sash servers such as MUDs), but there are several problems with this approach.

The first is the difficulty of choosing that port number 8080: it needs to be a number which no other program on the same machine has chosen, and it has to be known to the program or user who is trying to connect to this script. Sash solves this by using names for services instead of port numbers, and having a script called the broker manage the relationship between names and real port numbers.

The second problem is that this script has to be running when someone tries to connect to it, which in practice means it has to be running all the time: this is very untidy when there are many many scripts involved. Sash avoids this by having the scripts normally not running: the broker launches them as needed, they tend to wait around for a while after handling a request in case there's another one, but then exit until needed again.

The third is that when multiple requests arrive quickly they have to be handled one after another. If each handling takes a long time during which the host computer isn't really needed (for example if handling the request needs the server to fetch more information over the network or from slow disks), this delay is avoidable. Web servers get around this by running in multiple copies, but that's inflexible in needing the number of copies chosen in advance (mostly idle or not enough so there's still queueing), and more severely doesn't work for applications where the requests need to both read and write a common store of data. Sash deals with these problems by easy forking, in which the script splits into two copies as it runs, with one keeping just the server type variables active and the other keeping just the client type variables. This splits the 'management' function of accepting new requests into one process that owns the writable data and is rarely busy, from multiple 'worker' processes which actually deal with requests.

For these examples which depend on the sash broker you will need to have the broker running. It listens on the nesh/sash port 3507 which may have firewall implications if you're using a remote server.

#!/usr/bin/env sash
# while this script is running on your machine you should be able to connect from a web browser
# with URL http://localhost:3507/?service=sample
# this script must be a file called sample in the same directory as the broker

$listener = $servers.new
$client = $clients.new
$request = $strings.new

$listener.update =			# set a block of code to run when something connects
{
	$client = $listener.accept	# this variable holds specific client connection
	$request = $client.packet	# see what the client wants, but assume it's a webpage for now
	$client.content = "<h1>Simple Sash Web Server</h1>\n"
	# reply with web page, then close the connection
	# standard http headers are optional as broker will add them if needed
}

$listener.publish = "sample"		# use name of file (including path from broker directory)
:: 5000					# wait for 5 seconds (5000 milliseconds) handling requests
					# then exit
Similarly, sash client scripts can access sash services by name, for example:
#!/usr/bin/env sash

$service = $clients.new
$result = $strings.new

$service.target = "localhost:sample"
$service.packet = "give me the web page\n"
$result = $service.content

"Service returns:\n%s", $result
Although this is several times smaller than the equivalent code in other languages, it's verbose for sash because it takes 3 statements to do the common task of fetching the result of a request to a net service. This nuts-and-bolts style of communication is only needed for protocols like SMTP which have multiple exchanges of data between client and server. In most cases it's neater to use the 'fetch' attribute of a string containing the request, for example:
#!/usr/bin/env sash

$result = $strings.new

$result = "service=sample\n".fetch

"Service returns:\n%s, $result
The 'fetch' attribute uses its string to specify a network request, makes the request, and evaluates to whatever comes back from the net service. The request defaults to localhost if no 'server=name\n' line is included, and a 'port=n\n' line can be included instead of the service line for connecting to non-sash servers.

Sash forking is essentially unix forking, splitting the program into two copies which then run independently, except that one copy (the parent) keeps the open server connections and the other (the child) keeps the open client connections. The unix style fork uses an attribute of the '$system' variable which returns true in the parent and false in the child, as in ? $system.fork { # parent } !? { # child }, but this untidy and invites the bug of forgetting to exit the child process in the right place.

The neater form of forking is to enclose a code block in {{ }} instead of { } so that it runs as a forked process and exits at the end of the block. The basic skeleton of server script in this form looks like this:

#!/usr/bin/env sash

$listener = $servers.new
$client = $clients.new

$listener.update =
{
	$client = $listener.accept		# accept a new connection request
	? 1					# syntax glue as forked chunk must be a block
	{{					# fork to give it to a child process
		$system.sleep = 2000
		$client.packet = "1\n"
		$system.sleep = 4000
		$client.packet = "2\n"
		$system.sleep = 6000
		$client.packet = "3\n"
		$client.content = "All Done\n"
	}}
}

$listener.port = 4000				# use fixed port to allow telnet access
:: 0						# sleep, waking up to handle requests
Although each request takes 12 seconds to complete, the delays are all in child processes so the parent remains available to take new requests and fork new child workers: you can see this by making multiple overlapping connections with 'telnet localhost 4000'.

Strings

Sash string support includes the format strings and slicing functions of nesh and adds attributes. It's so powerful that programmers must avoid the trap of using strings instead of data structures, leading to code that is quick to write but too hard to maintain. Use these features to decode incoming strings, not to encode internal data.

Format strings have appeared above: they resemble the printf format strings of C but can be used as strings generally. Parameters can follow a string separated by commas and each is used to substitute into the string in place of the per-cent character and a letter. There are several exotic letters used in javascript generation, but the useful basics are as for printf: s = string, d = decimal number, c = character.

String slicing extracts part of a string, which can be chosen in several ways. The start and end points may be numbers which are indeces into the string, or subsstrings. The start and end points can be inclusive or exclusive of the number or string selecting it, and for each of these cases the start and end can be selected in different ways. Inclusive selection is shown by square brackets [] and exclusive by round brackets (). For example:

#!/usr/bin/env sash

$hello = $strings.new
$hello = "Hello World\n"

$hello[1 3] "\n"			# ell
$hello(1 3) "\n"			# l
$hello["Wo" "ld"] "\n"			# World
$hello("ll" "o") "\n"			# nothing
$hello(0 "ll") "\n"			# e
$hello["e" "d") "\n"			# ello Worl

Lists

Lists resemble arrays in other languages but they are searchable like associative arrays and fungible, like very little else. Searchable means any use of a list can be replaced by a sublist of just the elements which match some set of criteria. Fungible means that all elements of the list remain separate variables and can simultaneously be part of multiple lists. Deleting a variable removes it automatically from all the lists it was in, relieving the programmer of keeping track of some types of data relationships.

Lists mostly have different attributes to the variables they are made of, as seen above with the special list variables $integers, $strings etc. which are lists of all the script's variables of that type and can create more with the 'new' attribute. The attributes of variables in a list can be used to search it, by putting the test for inclusion into square brackets after the list variable.

Practical examples usually involve compound datatypes not covered yet, so we'll resort to pseudo-code:

# this is a fragment, not a runnable script

$my_ship = $ships.new
$other_ship = $ships.new
$my_ship = # the ship of the player currently being considered

$ships[star == $my_ship.star]		# the list of ships in the same place as this player
$ships[$current_turn - start_turn < 100 && energy > 10000]
					# ships younger than 100 turns who have at least 10000 energy

Lists can also be used in while loops, combined with a control variable by the '@' symbol so that the control variable becomes each variable in the list in turn. Note this is literally true, the control variable isn't a copy of the real variable: it is the variable so altering it will change the real variable and deleting it will remove it from the list, and all other lists.

?? $other_ship @ $ships[star == $my_ship.star]
{
	"%s is visible here, a %s class ship of size %d\n",
		$other_ship.name, $other_ship.class, $other_ship.size
}
!?			# while loops can have 'else' clauses for when they're not run even once
{
	"There are no other ships visible here\n"
}
and back to runnable scripts for a file example, using the list variable $files which is initially all the files in the current working directory, filtering to find only large files modified in the last 24 hours:
#!/usr/bin/env sash

$file = $files.new

?? $file @ $files[length > 1000 && date > $system.now - 24 * 3600]
{
	"File %s, size %d bytes, date %s\n", $file.filename, $file.length, $file.date.date_string
}

Files

File variables differ from integers and strings in several ways. They have several attributes which are really attributes of the files the variables represent, and the list variable $files is generally driven by the outside world rather than being a list of variables in the script.

The common file actions (create, read, write) use just the 'filename', 'content' and 'append' attributes. There are no separate create, open or incremental read commands: files are typically moved between mass storage and memory with a single assignment to or from a string, with all complex manipulation done on the string. For example:

#!/usr/bin/env sash

$file = $files.new
$data = $strings.new

$file.filename = "newfile.txt"
$file.content = "First line of file\nSecond line\nThird line\n"
# at this point the file is on disk with the above contents

$file.append = "Fourth line added at the end\n"
# additional data can be added only at the end

$data = $file.content			# read file contents back into memory

"file contains:\n%s", $data
The file list variable $files is initially a list of the files in the current directory (strictly it's a list of variables representing those files, but it's clearer to think of it as a list of files). A file list can be modified to be the files in another directory with the 'new_dir' attribute or to add the files from a different directory to the current list with the 'add_dir' attribute.

Images

An image type variable holds a picture in 24 bits, 8 each for red, green and blue components. The internal format is most directly convertable to ppm format, which is an easily manipulated but horribly verbose raster format. In practice images are usually converted via ppm to gif before being released into the outside world.

To make a simple red circle in a black square gif in a file called sample.gif:

#!/usr/bin/env sash

$image = $images.new
$file = $files.new

$image.width = 512			# set size of image
$image.height = 512
$image.background = "0 0 0"		# set background RGB components, all black and no foreground
$image.circle = "255 0 0"		# put in a red circle as big as possible

$file.filename = "sample.gif"
$file.content = $image.ppm.gif		# convert internal format to ppm, then to gif, and save to file

Manipulation of the image happens at a particular pixel, defined by its xdraw and ydraw attributes. These are used as the top left corner when embedding one image in another, as in this example of generating a random star field:
#!/usr/bin/env sash

$image = $images.new			# main image for star field
$star = $images.new			# small image to hold individual star
$file = $files.new
$count = $integers.new

$star.width = 10			# set up small star image
$star.height = 10
$star.background = "0 0 0"

$image.width = 512			# set up large star field image
$image.height = 512
$image.background = "0 0 0"

?? $count @ 1..10			# include 10 stars
{
        $image.xdraw = 500.die		# use 500 rather than 512 to avoid clipping at the edges
        $image.ydraw = 500.die
        $star.circle = "%d %d %d",	# make a star of random colour
		255.die, 255.die, 255.die
        $image.add = $star		# embed it by addition, so overlapping stars merge
					# the alternative is .insert, so overlaps replace
}

$file.filename = "sample.gif"
$file.content = $image.ppm.gif
Text can be written into an image at the drawing point using the .char (integer) or .text (string) attributes, but the font only currently contains digits and random punctuation marks.

For most precise control, the pixel attribute can be used on left or right side of assignments to write or read single pixels as a string of RGB values, eg "0, 128, 0" for mid-green.

Other Highlights

The switch statement 'keyword' is ???, which is followed by a control string and a {} block containing pairs of strings and code blocks. The first of these strings to match the control string has its code block executed. For example:
#!/usr/bin/env sash

$input = $strings.new

$input = "GET / HTTP/1.0\n\n"

??? $input
{
	"GET"
	{
		"It's a GET command\n"
	}
	"POST"
	{
		"It's a POST command\n"
	}
	"HEAD"
	{
		"It's a HEAD command\n"
	}
}
Integer ranges can be used instead of lists in while loops, written as two integer expressions separated by a pair of dots. The loop control variable then starts at the first value and increments on each loop until exceeding the second value. For example:
#!/usr/bin/env sash

$i = $integers.new

"13 times table:\n\n"

?? $i @ 1..12
{
	"%d x 13 is %d\n", $i, $i * 13
}

All list variables can be indexed by numbers, like arrays, but this is only rarely useful since other mechanisms are more powerful. The common example is for the pseudo-variable list $parameters, which is a list of the script's arguments. For example, a script called tel run with the command 'tel localhost sample' will have $parameters.1 set to 'tel', $parameters.2 set to 'localhost' and $parameters.3 set to 'sample'. The $parameters.0 entry is set to the name of the sash interpreter (for which these are of course argv).

Real Example Scripts

Here is the proxy script, which accepts connections and passes them on to a target hard-coded within it. Uses I've put variants of this script to include: logging network traffic to decode or debug net services, moving an unconfigurable legacy program to a different TCP port to avoid a firewall, modifying specific packets as they go by to experiment with changes or be able to pass overly strict checks.
#!/usr/bin/env sash

$listener = $servers.new
$listener.port = 4000
$client = $clients.new
$server = $clients.new
$data = $strings.new


$listener.update =
{
	$client = $listener.accept
	$server.target = "www.pbm.com:80"	# put target machine and or service here
	? $server.socket			# if no error, fork sleeping worker and continue
	{{
		:: 0				# proxy initiates nothing, just sleeps and reacts
	}}
}

$client.update =				# handler for incoming packet from client end
{
	$data = $client.packet			# put in variable for modifying or logging
	? $data
	{
		# could log data here by appending to logfile, eg $client_logfile.append = $data
		$server.packet = $data
	}
	!?					# if there's no data it must be a close from client
	{
		$server.content = ""		# close server end
		$system.exit = 0		# and exit worker child
	}
}

$server.update =				# mirror image of client handler above
{
	$data = $server.packet
	? $data
	{
		$client.packet = $data
	}
	!?
	{
		$client.content = ""
		$system.exit = 0
	}
}

:: 0						# parent process sleeps waiting for new connections
Note the use of 'update' attributes on the client variables, these blocks of code are run when something happens to their variable, either incoming data or a connection close from the other end.

Forking of the worker child allows multiple concurrent proxying by the same script. For example, web browsers fetching a complex page with pictures in usually run multiple connections to get the pictures and text simultaneously: a non-forking proxy would slow this down by waiting for each connection to complete before starting the next. Forking, sleeping and 'update' handling allows very complex behaviour to emerge from simple source code.


Attribute Reference Tables

Each attribute can only be used on right-hand-side of assignments unless noted as (LHS or Both)

Integers
AttributeTypeMeaning
absIntegerAbsolute value
atomStringName of a matching atom (like enum)
cosInteger1000 times cosine
date_stringStringTick count formatted as date and time
dieIntegerRandom number from 1 to subject
dot_quadStringa.b.c.d (ip address)
factorsStringFactors
idIntegerInternal id of subject variable (Both)
listList of IntegerList of just subject variable
maxIntegerlimit subject maximum to target (LHS)
minIntegerlimit subject minimum to target (LHS)
sinInteger1000 times sine
sqrtIntegerInteger square root
typeIntegerInternal constant for integer type (Both)
updateCodeCode to run when subject is modified (Both)
valueIntegerValue

Strings
AttributeTypeMeaning
addStringCombine strings with x=n lines arithmetically (LHS)
ampersandStringReplace newlines with ampersands
booleanStringInexplicable, needed for javascript compatibility
chopStringRemoves start of subject up to first occurrence of target (LHS)
content_lengthIntegerContent Length: number from http headers
content_typeStringContent Type: field from http headers (Both)
credentialsStringhttp header credentials decoded from base64 (Both)
deleteStringRemove target substring from subject (LHS)
executeStringRun subject as command in parent (usually non-sash) shell
fetchStringUse subject as net service request specification
formStringObsolete form generation primitive (LHS)
gifStringConvert ppm string to gif format string (needs ppmtogif)
headlineStringFirst line of subject
http_statusStringStatus from http headers (Both)
headlineStringFirst line of subject
idIntegerInternal id of subject variable (Both)
ipv4Integer32 bit network address of subject machine
lengthIntegerNumber of characters in string (no null terminate assumption)
listList of IntegerList of just subject variable
locationStringLocation field in http headers (Both)
lowerStringAll letters to lower case
mixedStringFirst letter of each word to upper case, rest to lower case
numberIntegerConvert string to decimal number
portStringManipulate port=x element of strings
quoteStringMake subject quotable by converting difficult characters to escape sequences
realmStringRealm field from http headers (Both)
requestStringManipulate request=x element of strings
resourceStringDecodes 'url' encoding to text
serverStringManipulate server=x element of strings
serviceStringManipulate service=x element of strings
stringStringValue of string
tidynameStringMap '_' to ' ' and '@' to ''' for TBG-1 compatibility
typeIntegerInternal constant for string type (Both)
upperStringAll letters to upper case
urlStringEncode for 'url' format (%2F+ etc.)

System
AttributeTypeMeaning
broker_portIntegerPort used by broker on this system, default 3507
dirStringCurrent working directory (Both)
exitIntegerEnd program, setting exit status (LHS)
forkStringFork program, returning true and false
hostnameStringName of host computer
idIntegerInternal id of subject variable (Both)
nowIntegerTime in seconds since The Beginning
nullStringDiscarding destination (LHS)
pidIntegerProcess ID
randomIntegerSet seed on LHS or read random number on RHS (Both)
sleepIntegerSleep for set number of milliseconds (LHS)
stdinStringRead from standard input
stdoutStringWrite to standard output (LHS)
typeIntegerInternal constant for system type (Both)
utimeIntegerTime of day in microseconds

Files
AttributeTypeMeaning
appendStringAdd data to the end of the file (LHS)
contentStringRead or write all the data in the file (Both)
dateIntegerDate of last modification in seconds since The Beginning
filenameStringName of file (Both)
idIntegerInternal id of subject variable (Both)
lengthIntegerSize of file in bytes
listList of IntegerList of just subject variable
typeIntegerInternal constant for file type (Both)

Images
AttributeTypeMeaning
addImageAdd one image to another, pixel by pixel, at xdraw, ydraw (LHS)
backgroundstringFill image with RGB colour from string (LHS)
bitmapstringSet image to contents of bitmap in X/c format (LHS)
charintegerDraw character at xdraw, ydraw (LHS)
circlestringPut biggest circle possible into image, using RGB string for colour
heightintegerSet or read height of image (both)
idIntegerInternal id of subject variable (Both)
insertimageInsert one image into another at xdraw, ydraw (LHS)
pixelstringRead or write pixel at xdraw, ydraw as RGB string (both)
pixel24integerRead or write pixel at xdraw, ydraw as 24 bit integer (both)
ppmstringRead or write image in ppm format (both)
roundelstringDraw roundel into image according to string value (LHS)
textstringDraw text string into image at xdraw, ydraw (LHS)
widthintegerSet or read width of image (both)
xdrawintegerSet or read x drawing position (both)
ydrawintegerSet or read y drawing position (both)

File Lists
AttributeTypeMeaning
add_dirStringAdd all files in another directory to current list
firstFileFirst file variable in list
lengthIntegerNumber of files in list
newFileCreate new file variable
new_dirStringReplace list with files in target directory
typeIntegerInternal constant for file list type (Both)


Maintained by Jeremy Maiden