Note
Make sure you installed FMS properly before following this tutorial, see FMS download and installation.
FMS is a command line application and as such, is rather self-documented. To get a summary of the various commands and options, cd to the directory where you installed FMS and simply run:
python startfms.py --help
You should get something like:
Usage: startfms.py [options] [command] simulationconffile
runs a Financial Market Simulator simulation
Options:
-h, --help show this help message and exit
-v, --verbose set logging level to 'info'. Overrided by --loglevel.
-L LEVEL, --loglevel=LEVEL
set logging level to LEVEL: debug, info, warning,
error, critical.
As you might see, you just used the first option, --help.
For further details about commands and options, an unix-style manpage is provided in the docs directory of your FMS installation.
“Running” FMS means feeding it with information about which kind of agents (and how many of them) trade on which kind of market in which kind of world, for how many periods of time. In addition, the market may follow different rules of clearing, depending on the “period” of the day (pre-opening, fixing, continuous trading, etc.). Such a complete description is called an “experiment” and is conveniently stored in a flat file (see FMS experiments configuration for details about this file). Once FMS read and checked the configuration file, the experiment is run, producing transaction data (date, transaction number, price, volume) either on the console or in a file.
Before going deeper in the nuts and bolts of FMS, you should understand what world, engines, markets and agents mean in FMS’s vocabulary.
This is a “global environment” class, providing agents with so called exogenous information on request. Such information might typically be the level of interest rates, or energy price, for an example. Any world class keeps track of time.
A NullWorld class is provided with FMS, it does not provide any special information.
The engines/markets tuples describe what you would simply call “the market” in the real world. Engines are the “traffic controlers” : they give speak to the agents in a (simulated) synchronous or asynchronous manner, choosing which agents speak and when at will. For an example, FMS provides with an AsynchronousRandWReplace engine class, which is asynchronous (market clearing is required as soon as an agent spoke) and chooses agents randomly, with replacement. Markets basically are responsible for recording the orders, and doing the clearing (for an example, auction style “fixing” clearing once in a while, or continuous book based clearing). FMS provides with two basic market classes, ContinuousOrderDriven and HighestQtyFixing.
Agents act when the engines give them speak. Acting is either do nothing, or place an order. Order should at least have a direction (buy or sell) but may in addition specify price and/or quantity. A ZeroIntelligenceTrader class is provided: this agent takes fully random decisions.
As all this probably sounds rather theoretical at the moment, let’s try a first experiment to help you make sense out of it.
To keep your system clean, start by creating a new directory for your experiments somewhere in your home directory (the My documents directory if you use Windows).
If you use Linux or a BSD, make sure that the startfms.py script is on your path, or symlink or copy it in the newly created “experiments” directory.
If you use Windows, copy the startfms.py (it is very small, so copying it is not a problem) in the newly created “experiments” directory. Remember that startfms.py is located in your Python installation Scripts directory, something like C:\Python26\Scripts\.
Open your favorite text editor, and copy the text below:
--- # My first experiment
world:
classname: NullWorld
engines:
- classname: AsynchronousRandWReplace
market:
classname: ContinuousOrderDriven
daylength: 20
agents:
- classname: ZeroIntelligenceTrader
number: 1000
money: 10000
stocks: 1000
args: [1000, 100]
Then save it in the “experiments” directory under the name “exp1.yml” or whatever name you may think of (and remember), provided you keep the ”.yml” extension. This extension is necessary because it tells FMS that the configuration file is a YAML one. YAML (Yet Another Markup Language) is nothing more than some kind of syntax, and this one was designed to be as human friendly as possible, so don’t be scared even if you have no previous experience of manually written config files.
Before trying to run this first experiment, you deserve a few words of explanation.
As said before, the full description of the experiment should be in the config file [1]. Because you read carefully the previous section, you know that to define an experiment, you should (at least) provide:
As you probably noticed, this is exactly what you just copied in the configuration file. As any YAML file, this file contains “documents”. The three dashes on the first line mark the beginning of the only document in our configuration file:
--- # My first experiment
Anything following the # on this line (and any other) is a comment.
Then we describe three “objects”: any non-indented line looking like:
object:
starts the description of a new object. The idented lines below this one define this object’s various attributes. We thus defined exactly three objects, namely world, engines and agents. More objects are possible, see FMS experiments configuration to learn about those.
As the three kind of objects we described here are the bare minimum to get a valid experiment, let’s get acquainted with any of them.
The simple world we use in this experiment is fully described with these two lines:
world:
classname: NullWorld
This means that the only necessary attribute for the world object is classname, that is the kind (class) of world you want to use in your experiment. Here, the classname is NullWorld, and actually you read above in The FMS model that FMS is provided with a very simple world class, which name is NullWorld.
The term class is a technical one, refering to something very precise in the Python programming language. If you feel curious about that, get a command prompt and cd to the fms directory, which is probably in the site-packages directory of your Python installation. In this fms directory you see subdirectories, conveniently named worlds, agents, and so on. Now cd to the worlds directory: you should see a bunch of files, and among those, a nullworld.py file. This is where FMS will look for the NullWorld class when it will need it, i.e. when we will run our first experiment.
Then we have a world, time to think about running a market. This is the engines’ job.
As you read above, what we call a “market” in the real life is a tuple engine/market on FMS. This design decision helps following the “don’t repeat yourself” programming rule, and should not bother you too much.
Did you notice the plural in “engines” ? Yes, you may use more than one engine/market tuple for an experiment. This allows for an example to mimic some markets where a pre-opening period finishes with the determination of a fixing price, and then continuous trading. But you may of course imagine whatever combination fits your (research) needs.
Here is the part of the config file related to engines and markets:
engines:
- classname: AsynchronousRandWReplace
market:
classname: ContinuousOrderDriven
daylength: 20
As you may have more than one engine/market tuples, the class names of the engines is in a list. This is why the classname block is preceded by a dash. The dash says “this is an item in a list”.
So, we only have one engine here. What are the engine’s attributes ? For the same reason why the world needed a classname attribute, so does the engines, and will the market and the agents. The classname says “use this kind of engine”. As any engine is associated with a market in a tuple, the classname attribute is followed by a market attribute. The market itself is an object, and (guess what) gets a classname attribute.
Finally, this precise engine, which classname is AsynchronousRandWReplace will run 20 times, as the daylength attribute value is 20. Note that this attribute is optional, but the default value is 1, which is not much. This attribute is not the number of transactions you will get in the output, but the number of times the engine will give speak to an agent. When an agent “speaks”, she may decide to do nothing, or place an order which does not necessarily trigger a transaction, depending on the market model. Thus, the number of times the engine give speak to the agents is not the number of transactions of the experiment.
The last object we put in the config file is the agents one. Its description is as follows:
agents:
- classname: ZeroIntelligenceTrader
number: 1000
money: 10000
stocks: 1000
args: [1000, 100]
We have here the same kind of block structure than the one we had with engines. An experiment may use different kind of agents (random, fundamentalists, ...), thus the classname is preceded by a dash, telling us that other agent classes might follow in the configuration file.
Here we use only one class of agents, namely ZeroIntelligenceTrader. This agent always take random decisions. The attributes block starts with number, which, as you guessed, gives the number of individual agents of that class you need.
Then come money and stocks, which describe the initial portfolio of the agent, in cash and stocks.
number, money and stocks are compulsory attributes, whatever the agent class you choosed.
The last attribute, args, is a list of optional attributes, the meaning of which depends on the agent class. For the ZeroIntelligenceTrader class, the attributes are the maximum price and quantity an agent is allowed to use in an order, i.e. these are limits for the random process generating the orders.
Your experiment is configured, it is now time to check that everything is ok. Again, cd to the “experiment” directory and run:
python startfms.py -v check name-of-the-config-file.yml
(Replace name-of-the-config-file with the name of your config file).
The -v option stands for “verbose”, as we want FMS to tell us everything it does. The output should be:
INFO - fms.utils - Reading config file name-of-the-config-file.yml
INFO - fms.utils - == name-of-the-config-file.yml experiment ==
INFO - fms.utils - Output file : sys.stdout
INFO - fms.utils - World : NullWorld
INFO - fms.utils - Engines :
INFO - fms.utils - AsynchronousRandWReplace : 1 days of 20 instants on ContinuousOrderDriven market
INFO - fms.utils - Agents :
INFO - fms.utils - 1000 ZeroIntelligenceTrader with $10000.00 and 1000 stocks, [1000, 100]
# name-of-the-config-file.yml experiment
time;transaction;price;volume
INFO - fms.utils - Config file name-of-the-config-file.yml parsed.
INFO - fms - Created world fms.worlds.nullworld.NullWorld world 162324140
INFO - fms - Created engine-market fms.engines.asynchronousrandwreplace.AsynchronousRandWReplace engine 162308012 - fms.markets.continuousorderdriven.ContinuousOrderDriven market 162320972
INFO - fms - Created 1000 instances of agent fms.agents.zerointelligencetrader.ZeroIntelligenceTrader
As expected because we choose the “verbose” option, the output is copious. Actually, all the lines starting with INFO would not be here without the -v. Try it:
python startfms.py -v check name-of-the-config-file.yml
You should get:
# name-of-the-config-file.yml experiment
time;transaction;price;volume
This is the only “real” output, that is, the header of the transaction file, because no actual transaction occured. That is because we used the check command, which is a dry-run: it does everything a real run would do (including the output of a header, thus checking that writing on the output device, console or file, is possible), but stops just before starting the (first) engine.
Read carefully the INFO lines in the output above: FMS first informs you that it reads your experiment configuration file. Then, it tells you what it understood from what you write there, one item at a time. You know when the file is completely parsed, and this is the moment FMS tries to create all the objects needed to complete your experiment. You can read about these operations too : what it created, and how many of these objects.
Ok, here all went well, we could run the experiment and see the output right now. But wait, what if we did a mistake in the configuration file ?
Edit the configuration file again, and change the fourth line so that the world block is now:
world:
classname: OtherWorld
Save, and run the check again (you know how to do this, don’t you ?). The output (without the verbose option) should be:
# name-of-the-config-file.yml experiment
time;transaction;price;volume
CRITICAL - fms - Unknown fms.worlds class: OtherWorld
FMS read the config file, output the result header, then tried to create an OtherWorld world. As this class does not exist so far, it failed, and as a world class is necessary for the experiment to run, this is a critical situation. FMS thus stops immediately, and informs you about this, even if you did not choose the verbose option.
Let us try something nastier. Edit again the world block in the configuration file, correct the class name but change the attribute name:
world:
theclassname: NullWorld
Save, check. You should get something like:
ERROR - fms.utils.exceptions - Missing parameter: world classname
None
Traceback (most recent call last):
File "startfms.py", line 161, in <module>
sys.exit(main())
File "startfms.py", line 147, in main
params = YamlParamsParser(simconffile)
File "fms/utils/__init__.py", line 336, in __init__
raise MissingParameter, 'world classname'
fms.utils.exceptions.MissingParameter: world classname
Ouch, this looks scary. But look again and see the first line: FMS is kind enough to tell you something understandable before crashing:
ERROR - fms.utils.exceptions - Missing parameter: world classname
And this exactly the problem. We created a theclassname attribute to the world object, and there is nothing wrong with this (well, it is useless, but harmless too). But there is no classname attribute, and FMS needs a world classname as a parameter for the experiment. The missing parameter triggers an error, and FMS reports about the error (and actually does not crash, but stops, outputting complete exception information in case someone needs it).
Now you learned that:
Now that we checked that the experiment configuration file is correct, we may run the experiment for real. This is simply done by entering the run command instead of the check one we used so far:
python startfms.py run name-of-the-config-file.yml
Here is the output on my computer:
# name-of-the-config-file.yml experiment
time;transaction;price;volume
1;1;900.68;21
5;2;216.18;14
5;3;643.92;61
8;4;137.74;26
8;5;218.66;51
9;6;218.66;62
14;7;218.13;25
15;8;218.13;28
16;9;218.13;27
17;10;218.13;35
The output you get will probably be different: this is because this experiment uses a pseudo-random number generator, and we did not feed it with a seed. In this situation, the seed is “chosen” by the system, and the result is unpredictable, depending on the OS you use, the time you ran the experiment, and many other parameters. If you prefer to choose the seed yourself, you may add a randomseed object to the configuration file, giving it whichever value you want (as usual, see FMS experiments configuration for a complete description of what you may put in the configuration file).
Should you need details about the pseudo-random number generator, see the Python language library documentation.
What did we get as an output ? Hopefully this is transaction data. Each line (except the two header ones, that is) describes a transaction as semicolon separated numerical values. As described on the second header line, you get:
Again, the experiment lasted 20 turns, that is the engine class gave speak to a randomly chosen agent 20 times before stopping. More precisely, what happened twenty times is:
Given the random value chosen, this resulted in the 10 transactions above.
Naturally you will probably use far more data for your research, asking for many more turns. Notice that, in addition to daylength you may specify a days object, that is the number of days of daylength you want to run. You would then get a days * daylength turns experiment. This allows for fine tuning the experiment: some engines would for an example flush the order books at the end of any day, starting again with empty ones on the next day. By the way, this behaviour is so common that it is the default one.
Note that we did not ask FMS to be verbose here: this is because the output goes to the console (we did not put the name of an output file in the configuration), and I did not want to mix it with the information reported by FMS about what is going on when running the experiment. But as you will later use the transaction data to produce graphics or statistics or whatever you may imagine, it is convenient to store it in a file. The outputfilename object in the config file lets you give the name of the file storing the results of the experiment.
Let us try it. Edit again the configuration file and add this line somewhere it would not interfere with the other parameters (i.e., not right in the middle of another object’s block):
outputfilename: experiment-one.csv
Then run the experiment again:
python startfms.py run name-of-the-config-file.yml
You should get no output at all: we did not ask FMS to be verbose, and all the transaction data went to the experiment-one.csv file (in the current directory if you did not give a different path). Open this file with a text editor, and you should see the transaction data (a different set, because the random seed was different), each transaction described on a line exactly as it was output on the console in the previous run.
As we explained above, the transaction data on the second run is different from the one we got on the first run. This is because the randomly chosen random seed was different, thus resulting in a completely different sequence of orders.
Nonetheless, it may sometimes prove useful to “replay” an experiment, that is, having the same agents put the same orders at the same time. I use it a lot for testing purposes, for an example.
Because we used random agents in our experiment, this could be achieved by choosing the random seed ourselves, as a given random seed will always result in the same pseudo-random sequence of numbers. But this trick would not do for non-random agents. In addition, it might be useful to store the sequence of orders put by the agents, too. Thus, FMS is provided with the necessary set of tools to achieve both results:
The first step is achieved by another configuration item, namely orderslogfilename. As with ouputfilename, you simply provide a file name, and FMS will gently store the orders put by the agents in it.
The second step involves using a special agent class, PlayOrderLogFile: instead of speaking randomly, or being a fundamentalist, or whichever scheme we may imagine, this agent “reads” class the orders from the orderslogfilename, in sequence. In addition, to make sure the sequence is correct whatever the number of instances of this agent you ask for, each instance will actually be the same agent [2] (thus “remembering” which line of the file it just read, for an example).
To try this, we first need to play the experiment again, this time storing the orders the agents emitted. We will then replay the experiment and check that the output transactions are actually the same.
Edit your config file again, and add an orderslogfilename item:
orderslogfilename: myorders.txt
Run the experiment again, and check that there is a myorders.txt file in the current directory (or whichever path you choosed). Open this file with your editor and check its contents. You may notice that it contains exactly 20 order lines, as you asked for a daylength of 20. Close the file, and be careful not to change anything in it (do not save).
Then, change the configuration file again. There is a little bit of work to do here (you might prefer to copy the original configuration file, and modify the copy):
The resulting file contents should be like:
--- # My first experiment
world:
worldclassname: NullWorld
classname: NullWorld
engines:
- classname: AsynchronousRandWReplace
market:
classname: ContinuousOrderDriven
daylength: 20
agents:
- classname: PlayOrderLogFile
number: 1000
money: 10000
stocks: 1000
args: ['myorders.txt']
outputfilename: experiment-replayed.csv
Then run the experiment against the new configuration file. If you did not choose to pass the verbose options, you should get no output. Compare the contents of the original output file (experiment-one.csv) with the one resulting from the replay of the experiment (experiment-replayed.csv). If all went well (and there is no reason it did not), they should be identical. If your system is an unix-based one, you may try:
diff experiment-one.csv experiment-replayed.csv
As the diff comment produces no output at all [3], the two files contents are the same.
Congratulations ! You managed to go through this long tutorial, and are now ready to start playing with FMS on your own. Remember to read the rest of the documentation, it will help you master the fine-tuning aspects of FMS, getting the results you need for your research.
Happy number crunching !
Footnotes
| [1] | A future release of FMS might allow to pass all the parameters as options, for people loving long and convoluted command lines... |
| [2] | Technically, this kind of class is called a “Borg”, after a fictional pseudo-race of cyborgs depicted in the “Star Trek” series. The Borgs have some kind of collective mind, as our agents share the same state. |
| [3] | Actually it may produce output if you changed the experiment configuration file name, as it is used to produce the first line of header. |