# KEHOME/doc/MKRtoken.html
# Aug/12/2003
MKR tokens

MKR tokens
version 6.1

# The MKR parser has three phases
#
# 1. token.icn: get_word(): identify low-level word classes, e.g.
#	b	white space
#	s	separator
#	w	word
# 2. token.icn: get_token(): identify special words and operators, e.g.
#	A	at
#	H	has isin
#	D	do
#	o	od
#	d	done
#	R	relation: isa isa* isa**n ...
#	P	preposition: of with from to out
#	=	assignment operator: =,+=,-=,*=
# 3. symbol.icn: get_symbol(): identify phrases, sentences, ..., e.g.
#	www;	word phrase
#	wHw=w;	subject has attribute=value
#	wRw;	subject rel object
#	wiwPw=w;subject is  object with attribute=value


#------------------------------------#
# Here's the complete list of tokens #
#------------------------------------#

B	white space: blank tab return newline newpage
N	non separator: ~ / \ ? . $  (included in w)
W	weak separator: + - * :     (included in x)
S	strong separator: , ; = ! & | [ ] { } ( ) < > " ' #
w	word: consecutive bytes not B nor S nor W
x	extended word: consecutive w or W

"	quote mark: " or '
q	quote
#	comment mark: #
c	comment

# NOTE:
# For high level parsing (phase 3 in symbol.icn),
# the types above are remapped to
#	B	=> deleted
#	W	=> individual characters
#	S	=> individual characters
#	w	=> w
#	x	=> w
#	"	=> not used
#	q	=> w
#	#	=> not used
#	c	=> deleted


,	list separator: phrase
;	list separator: sentence, proposition

~	OK in word: not
:	OK in word: view-concept, class-number, unit
+	OK in word
-	OK in word
*	OK in word: wildcard

A	at: context -- space,time,view
B	user-defined binary relation
C	rdfs:Class
D	do do* vdo
d	done
E	else
F	fi
G	generator: every,when,while
g	group: ntriple,mcf,rdf,... (use begin/end)
H	has isin haspart isapart
h	hierarchy (use begin/end)
I	if
i	is isa iss isu isd isi
J	conjunction: ";" iff implies causes means
j	conjunction: and or
N	complement: not non
P	preposition: out of with from to od at
p	rdf:Property
Q	quantifier: a some the no all any either
R	relation verb: isa* isa** ... isc isc* isc** ... in ...
r	relation (use begin/end)
S	assignment: set vset unset
s	space
T	then
t	time
U	unknown
v	view
w	word not predefined by MKR
x	xml,rdf
Y	group definition: begin
Z	group definition: end
z	exit

()	grouping: not currently used
[]	begin/end of comma list: phrase
{}	begin/end of semicolon list: proposition
<>	embedded HTML

/	down one level in hierarchy
\	up one level in hierarchy
?	question
!	sh command
$	value of variable
$()	product (a la KornShell) -- not currently used
..	dot variable: current genus
.	dot variable: current concept
...	dot variable: current species

=	assignment operator =
=	Set add operator +=
=	Set delete operator -=
=	Set intersection operator *=
=	production operator :=
=	Backus-Naur Form grammar operator ::=


&	and, asynchronous execution (not implemented)
|	or,  pipe (not implemented)
~	not, login directory


##### possible additions #####
m	method
o	od: direct object
u	unit

MKE home