With SelectScript we wanted to implement a general interface for various simulation environments. Retrieving data, information, and also abstractions from a simulation environment should be as easy as querying a database with SQL, without paying attention to the underlying interfaces ... The following video shall give a little impression about the usage of SelectScript. This version, called SelectScript_OpenRAVE, was especially developed for OpenRAVE, but as we will show, it is easy to adapt the language and to integrate new functionalities ... Because our query language is under steady development, we will adapt also this tutorial step by step to allow more insights...
from IPython.display import YouTubeVideo
YouTubeVideo('R_PThP0gwOc', width=640, height=430)
If you have comments, suggestions, hints, etc. (or even problems), feel free to contact us.
André Dietrich, dietrich@ivs.cs.uni-magdeburg.de, http://eos.cs.ovgu.de/dietrich/
Sebastian Zug, zug@ivs.cs.uni-magdeburg.de, http://eos.cs.ovgu.de/zug/
This extension of SelectScript for OpenRAVE was implemented in Python2.7 and with the help of ANTLR3 (everything was tested on Ubuntu 12.04). There are different ways to install SelectScript_OpenRAVE, but first of all you will have to install OpenRAVE itself on your system, follow the installation instructions at http://openrave.org/docs/latest_stable/install/#install, we recommend to use the latest stable version 0.9.0. Afterwards follow one of the 3 steps listed below:
We recommend the usage of the python install tool (pip), which automatically downloads and installs all required sources and dependencies.
pip install SelectScript_OpenRAVE
Download the ta.gz from: https://pypi.python.org/packages/source/S/SelectScript_OpenRAVE/SelectScript_OpenRAVE-1.tar.gz
and install it manually with: python setup.py install
Checkout our project from http://sourceforge.net/projects/selectscript/
svn checkout svn://svn.code.sf.net/p/selectscript/code/trunk/ selectscript
This will download the whole implementation of SelectScript containing the base implementation, grammar files, and other derived implementations for different simulations, etc. Afterwards install the packages according to section 2.2. This will be the most frequently updated version ...
We added some basic examples (demo0.py, demo1.py, demo2.py, demo3.py, demo4.py) to demonstrate and present the aspects of our approach. All the demos can be started with the following 2 lines of code ... (you should see a running simulation at this point, or something with the installation went wrong)
from SelectScript_OpenRAVE.examples import demo0
demo0.start()
To initialize all required modules run the following commands, which will load OpenRAVE, some further requirements as well as the SelectScript parser and the interpreter for OpenRAVE (ssRAVE).
from openravepy import *; import numpy, os
from SelectScript.SelectScript import SelectScript
import SelectScript_OpenRAVE.interpreter as ssRAVE
Initialize a new OpenRAVE environment and loud our kitchen-example:
env = Environment() # create the environment
env.Load(os.path.dirname(ssRAVE.__file__)+"/examples/resources/kitchen.env.xml") # load a model
Afterwards initialize the SelectScript parser and then the interpreter. The parser is the same for every language implementation and generates some kind of simplified byte code, which is easier to interpret. But the interpreter itself has to be adapted, to work with different simulations environments; we will describe this later in more detail.
ssPar = SelectScript(None, None)
ssInt = ssRAVE.interpreter()
To start the visualization run the following command.
env.SetViewer('qtcoin') # start the viewer
We will use an additional feature of OpenRAVE to place screenshots of the simulation into this notebook. The following code is only used for visualization purposes ...
%pylab inline
import matplotlib.pyplot as plt
plt.rcParams['figure.figsize'] = (10.0, 8.0)
def screenshot():
env.GetViewer().SendCommand('SetFiguresInCamera 1')
I = env.GetViewer().GetCameraImage(640,480, env.GetViewer().GetCameraTransform(),[640,640,320,240])
plt.imshow(I)
plt.show()
screenshot() # show a screenshot of the environment ...
SelectScript is intended to be used for queries only, that is why you can call only externally defined functions (parameters have to be separated with commas), apply basic arithmetic and logic operations, store values in variables, and last but not least define queries (similar to SQL). Every expression has to end with a semicolon and parenthesis is supported.
expr = "1+2*3-4/5.;" # expression is defined within a string, supported datatypes are bool, int, float, string, list
prog = ssPar.compile(expr) # some bytecode is generated
print ssInt.eval(prog) # and interpreted ...
expr = """ 2 * 2; // multible commands can defined within a script, comments are in C style ...
a = False; """ # the last expression of a script defines the return value of the whole script
prog = ssPar.compile(expr)
print ssInt.eval(prog)
expr = """ b = (2+2)*3.45;
a = False or True;
[b, a, "a string", 'or another string']; """ # use lists to define more complex return values ...
prog = ssPar.compile(expr)
print ssInt.eval(prog)
Variables are stored persistently within the SelectScript interpreter and can also be called from outside with the following methods.
print ssInt.callVariable('b')
ssInt.addVariable("c", 1.234) # attach a new variable
print ssInt.callVariable('c')
The next parts will describe how to use functions and of course, how to use queries and how to extend the language according to different requirements.
Functions are mostly used to define relations, to get a first overview on available functions simply call help() the result is a list of all defined functions so far ...
expr = "help();"
prog = ssPar.compile(expr)
ssInt.eval(prog)
To get help for a specific function-call use it in the following way.
expr = "help('isKinbody');"
prog = ssPar.compile(expr)
print ssInt.eval(prog)
expr = "help('obj');"
prog = ssPar.compile(expr)
print ssInt.eval(prog)
Before we can query the simulation environment, we have to add under a specific name to the interpreter. This is done in the same way as you would attach any other ordinary variable...
# add the environment as variable to the interpreter
ssInt.addVariable('kitchen', env)
Starting with a simple query, where we might only interested in the positions and identifiers from entities within the kitchen environment that fulfill the property of being a robot. The result is given back as list of python dictionaries, similar to a table in databases.
expr = "SELECT id, position, type FROM kitchen WHERE isRobot(this);"
prog = ssPar.compile(expr)
for element in ssInt.eval(prog):
print element['id'], ": \t", numpy.round(element['position'],3), "\t", element['type']
The return value of a SELECT query can be defined with the key word AS. Further supported abstractions are value and list, the python dict is default and does not have to be defined ...
expr = """// stores the table objects within a variable
table = SELECT obj FROM kitchen WHERE id(this) == 'table' AS value;
// select all volumes (AS list) of objects that are above the object
// table an with an volume smaller than 0.001
SELECT volume
FROM kitchen
WHERE above(table, this) and volume(this) < 0.001
AS list;
"""
prog = ssPar.compile(expr)
ssInt.eval(prog)
screenshot()
Something similar can be also done in the opposite way ...
expr = "SELECT obj, volume FROM kitchen WHERE below(table, this) AS dict;"
prog = ssPar.compile(expr)
ssInt.eval(prog)
Or to check which objects are within another object ...
expr = "microwave = SELECT obj FROM kitchen WHERE id(this) == 'microwave' AS value; \
SELECT obj, position FROM kitchen WHERE within(microwave, this) AS list;"
prog = ssPar.compile(expr)
print ssInt.eval(prog)
screenshot()
With the following Python command we simply switch on the lights or in other words, switch on the sensory systems and start to render the sensory measurements.
for sensor in env.GetSensors():
sensor.Configure(Sensor.ConfigureCommand.PowerOn)
sensor.Configure(Sensor.ConfigureCommand.RenderDataOn)
screenshot()
To check which of those sensors are currently measuring a robot, we can apply the following select query. The function to in this example is only used to translate the function name and thus the key of the dictionary from id to sensor and robot. This query simply evaluates the cross product of kitchen and kitchen, which is translated into e1 and e2.
Another remarkable aspect in this query is the new keyword this which is applied to place objects into the appropriate place, if functions are called with more than one parameter. So far there was no need for a this pointer, because all called functions required only one parameter, which is automatically substituted with this. For example the function-call volume after a select statement can also be written as volume(this). And if we use more than one "list" have to apply the appropriate name in front of the this pointer, such as e1.this and e2.this.
expr = "SELECT to(id(e1.this), 'sensor'), to(id(e2.this), 'robot') \
FROM e1=kitchen, e2=kitchen \
WHERE isSensor(e1.this) AND isRobot(e2.this) AND isSensing(e1.this, e2.this) \
AS dict;"
prog = ssPar.compile(expr)
ssInt.eval(prog)
The same query can also be simplified with multiple queries, which in the first case queries only for sensor systems and stores the results within a list structure. The same is done for robots afterwards, before these results are combined within the last query.
expr = "sensor = SELECT obj FROM kitchen WHERE isSensor(this) AS list; \
robot = SELECT obj FROM kitchen WHERE isRobot(this) AS list; \
\
SELECT to(id(sensor.this), 'sensor'), to(id(robot.this), 'robot') \
FROM sensor, robot \
WHERE isSensing(sensor.this, robot.this) \
AS dict;"
prog = ssPar.compile(expr)
ssInt.eval(prog)
Or as another possibility, we can simply place queries wherever we want ...
expr = "SELECT to(id(e1.this), 'sensor'), to(id(e2.this), 'robot') \
FROM e1=(SELECT obj FROM kitchen WHERE isSensor(this) AS list), \
e2=(SELECT obj FROM kitchen WHERE isRobot(this) AS list) \
WHERE isSensor(e1.this) AND isRobot(e2.this) AND isSensing(e1.this, e2.this) \
AS dict;"
prog = ssPar.compile(expr)
ssInt.eval(prog)
The following examples is used to demonstrate how both languages Python a SelectScript can be appropriately combined. At first we might be interested in defining a new relation (a function) to the interpreter that checks whether an object is placed on another. The following function can be used to check this, whereby the programmer has to take care about the data types. o1 and o2 have to be objects of type kinbody or robot, otherwise the call of this function will cause an error. Therefore, be aware to use on only with the appropriate data types or insert some additional checks ...
def on(o1, o2):
if o1 == o2: return False
ab = o1.ComputeAABB()
pos = o2.GetTransform()[:3,3]
if ab.pos()[0] - ab.extents()[0] <= pos[0] <= ab.pos()[0] + ab.extents()[0]:
if ab.pos()[1] - ab.extents()[1] <= pos[1] <= ab.pos()[1] + ab.extents()[1]:
if pos[2] >= ab.pos()[2] + ab.extents()[2]:
return True
return False
Attaching this function to the SelectScript interpreter is rather simple. Use the following method and include some additional information about the appropriate usage.
ssInt.addFunction('onX', # a name for the function
on, # the function pointer
# some helpfull information
"""Checks if the position of an object (o2) is above object (o1).
Usage: onX(o1, o2) -> bool """)
Afterwards it should also be possible to get some help about the newly defined function.
expr = "help('onX');"
prog = ssPar.compile(expr)
print ssInt.eval(prog)
With the next query, we can simply identify all relevant parameters, which are required to fulfill the job of cleaning up the table.
robot = env.GetRobot("barrett_4918")
ssInt.addVariable('robot', robot)
expr = "sink = SELECT obj FROM kitchen WHERE id(this) == 'sink' AS value;"
expr += "table = SELECT obj FROM kitchen WHERE id(this) == 'table' as value;"
expr += "cleanUp = SELECT obj FROM kitchen WHERE onX(table, this) AND distance(robot, this) < 1.3 AS list;"
prog = ssPar.compile(expr)
ssInt.eval(prog)
The example below shows how the results of a query can be easily applied within the program to solve the task of cleaning up the table...
from openravepy.examples.graspplanning import * # import the possibility to grasp
grasp = GraspPlanning(robot)
grasp.graspables = []
for object in ssInt.callVariable('cleanUp'):
destination = grasp.setRandomDestinations( [object],
ssInt.callVariable('sink'),
randomize=True, preserverotation=False)
gModel = databases.grasping.GraspingModel(robot=robot, target=object)
if not gModel.load():
gModel.autogenerate()
destination = [x.tolist() for x in destination[0]]
grasp.graspables.append([gModel,destination])
grasp.performGraspPlanning(withreplacement=True)
The video below shows the whole cleanup run ...
YouTubeVideo('jSaoCXRNVNg', width=640, height=430)
In some cases it might be useful to execute a specific function if a situation has changed, or in other words, if the result of a SelectScript has changed. The callback from below simply receives the result of a SelectScript query and stores the results within a global variable.
cbResult = []
def callback(result):
global cbResult
cbResult = result
The script below was taken from the examples in section "Complex Queries". The only difference is, that we do not execute the program directly but add trigger, which is fired if the result of the query changes and thus executes function callback, whereby the input parameters are always the result of a query.
expr = "env = kitchen; \
SELECT to(id(e1.this), 'sensor'), to(id(e2.this), 'robot') \
FROM e1 = env, e2 = env \
WHERE isSensor(e1.this) AND isRobot(e2.this) AND isSensing(e1.this, e2.this) \
AS dict; "
prog = ssPar.compile(expr)
ssInt.addTrigger('sensing_sensors', prog, 1, callback)
The both examples below simply show the changing results of triggered query...
print cbResult
screenshot()
print cbResult
screenshot()
The same technique was applied to a random walk of both robots. The callback function is executed whenever the set of currently measured robots changes. The callback function in demo2.py only switches on the rendering of sensors that currently perceive a robot.
YouTubeVideo('Bk8TaOQQdZM', width=640, height=430)
Last but not least, by developing SelectScript, we also wanted to enable developers to be able to attach own abstraction, which are defined with the keyword AS at the end of every select query. But doing this is a bit more complicated, it requires to extend the interpreter class. In the following three examples, we show that the possibility for abstractions is quite big.
The first abstraction we are going to define is a simple sub scene. The implementation can happen as follows.
class interpeter_environment(ssRAVE.interpreter):
# a new child class of the previous SelectScript interpreter for OpenRAVE
def __init__(self):
ssRAVE.interpreter.__init__(self)
# extend the evalAS method, if the keyword is environment, we will handle
# this locally, or otherwise call the parent class ...
def evalAs(self, AS, SELECT, FROM, FROM_n, WHERE):
if AS == "environment":
return self.environment(AS, SELECT, FROM, FROM_n, WHERE)
return ssRAVE.interpreter.evalAs(self, AS, SELECT, FROM, FROM_n, WHERE)
# this method simply creates a new environment with all entities that
# fulfill the defintions within the WHERE clause
def environment(self, AS, SELECT, FROM, FROM_n, WHERE):
mark = True
newEnv = None
for elem in FROM:
if not newEnv: newEnv = elem[0].GetEnv().CloneSelf(1|8)
if WHERE != []:
mark = self.eval(WHERE, elem, FROM_n)
if not mark:
object = self.eval(SELECT, elem, FROM_n)
if type(object) == Robot:
newEnv.Remove(newEnv.GetRobot(object.GetName()))
elif type(object) == KinBody:
newEnv.Remove(newEnv.GetKinBody(object.GetName()))
return newEnv
The rest is straight forward ...
ssIntX = interpeter_environment()
ssIntX.addVariable('kitchen', env)
## Example 1: Query for information about sensors and robots
expr = "roomba = SELECT obj FROM kitchen WHERE id(this)=='roomba_625x' AS value;"
expr += "SELECT obj FROM kitchen WHERE distance(roomba, this) <= 1.5 AS environment;"
prog = ssPar.compile(expr)
newEnv = ssIntX.eval(prog)
#newEnv.SetViewer('qtcoin')
See the running example from demo3.py at the YouTube video below ...
YouTubeVideo('k5NXVv6O3tU', width=640, height=430)
Something totally different might be a map for a certain area.... The example from above uses extends therefor the previous class and attaches a new filter. To run this example, you will have to install our filter-plugin for OpenRAVE. This plugin can be found at the following repository: http://code.google.com/p/eos-openrave-plugins/
class interpeter_map(interpeter_environment):
def __init__(self):
interpeter_environment.__init__(self)
RaveInitialize()
RaveLoadPlugin("/home/andre/Workspace/Projects/ROS/eos-openrave-plugins/filter/lib/filter")
def evalAs(self, AS, SELECT, FROM, FROM_n, WHERE):
if AS == "roombaGrid":
return self.roombaGrid(AS, SELECT, FROM, FROM_n, WHERE)
return interpeter_environment.evalAs(self, AS, SELECT, FROM, FROM_n, WHERE)
def roombaGrid(self, AS, SELECT, FROM, FROM_n, WHERE):
# only for visualizing:
env = FROM.next()[0].GetEnv()
if WHERE == []:
newEnv = FROM.next()[0].GetEnv()
else:
newEnv = self.environment(AS, SELECT, FROM, FROM_n, WHERE)
envmin = []
envmax = []
for b in newEnv.GetBodies():
ab = b.ComputeAABB()
envmin.append(ab.pos()-ab.extents())
envmax.append(ab.pos()+ab.extents())
envmin = numpy.floor(numpy.min(numpy.array(envmin), 0)*100.) / 100
envmax = numpy.ceil( numpy.max(numpy.array(envmax), 0)*100.) / 100
size = numpy.ceil((envmax - envmin) / 0.025)
ogMap = RaveCreateModule(env,'occupancygridmap')
ogMap.SendCommand('SetTranslation %f %f 0.22' % (envmin[0], envmin[1]))
ogMap.SendCommand('SetSize %i %i 0.025' %(size[0]+1, size[1]+1) )
ogMap.SendCommand('SetLineWidth 2.0')
render = ogMap.SendCommand('Scan')
ogMap.SendCommand('Render')
return numpy.fromstring(render, dtype=bool, count=-1, sep='').reshape(int(size[0]+1), int(size[1]+1))
Instantiate this new interpreter class ...
ssIntXX = interpeter_map()
ssIntXX.addVariable('kitchen', env)
And generates some maps ...
## Example 1: Generate a map from the enitre environment
expr = "SELECT obj FROM kitchen AS roombaGrid;"
prog = ssPar.compile(expr)
map = ssIntXX.eval(prog)
# plot the map
plt.imshow(map.astype(numpy.float), cmap=plt.cm.gray)
plt.show()
## Example 2: Generate a map from all objects that are close to roomba
expr = "roomba = SELECT obj FROM kitchen WHERE id(this)=='roomba_625x' AS value;"
expr += "SELECT obj FROM kitchen WHERE distance(roomba, this) <= 1.5 AS roombaGrid;"
prog = ssPar.compile(expr)
map = ssIntXX.eval(prog)
plt.imshow(map.astype(numpy.float), cmap=plt.cm.gray)
plt.show()
See the video below to get an impression, how our filters are applied to the simulation...
YouTubeVideo('MYQppe9E9Es', width=640, height=430)
It might sound strange, but with a little piece of code, we can also translate the simulation into a set of prolog clauses.
class interpeter_prolog(ssRAVE.interpreter):
def __init__(self):
ssRAVE.interpreter.__init__(self)
def evalAs(self, AS, SELECT, FROM, FROM_n, WHERE):
if AS == "prolog":
return self.prolog(AS, SELECT, FROM, FROM_n, WHERE)
return ssRAVE.interpreter.evalAs(self, AS, SELECT, FROM, FROM_n, WHERE)
def prolog(self, AS, SELECT, FROM, FROM_n, WHERE):
results = set()
mark = True
for elem in FROM:
if WHERE != []:
mark = self.eval(WHERE, elem, FROM_n)
if mark:
for f in SELECT:
if f[1] == 'to' :
result, relation = self.eval(f, elem, FROM_n)
pp = f[2][0][2]
else:
relation = f[1]
result = self.eval(f, elem, FROM_n)
pp = f[2]
if result != False:
clause = relation + "("
for p in pp:
p = self.eval(p, elem, FROM_n)
if isinstance(p, (openravepy_int.Robot, openravepy_int.Sensor, openravepy_int.KinBody)):
p = p.GetName()
clause += str(p) + ","
clause = clause[:-1] + ")"
if result != True:
clause = clause[:-1] + ","+str(result)+")"
results.add(clause)
return sorted(results)
A query afterwards can look like this
ssRave = interpeter_prolog()
ssRave.addVariable('kitchen', env)
expr = "SELECT above(a.this, b.this), \
isEnabled(a.this), \
volume(a.this), \
position(a.this), \
isRobot(a.this), \
isKinbody(a.this) \
FROM a=kitchen, b=kitchen \
WHERE not (isSensor(a.this) or isSensor(b.this)) \
AS prolog;"
prog = ssPar.compile(expr)
clauses = ssRave.eval(prog)
To show that this works, we can open an interface to SWI-Prolog and attach the retrieved facts with the example below...
from pyswip import Prolog
prolog = Prolog()
for result in clauses:
print result
prolog.assertz(result)
And start to query in Prolog syntax ...
print list(prolog.query("above(table, X)"))
print list(prolog.query("volume(Obj, V), V > 1.0"))
print list(prolog.query("position(_, P)"))