Downloads page of Jose Antonio Martin H

Jose Antonio Martin H. PhD Madrid, España
"Si creen que la investigación y la educación son caras, prueben con la ignorancia y la mediocridad". Joan Guinovart.

Third Annual Reinforcement Learning Competition 2009. it has just ended now, join us next year!.

Competition Records for JAMH Team - Reinforcement Learning Competition

1st First place PolyAhtlon 2008
2nd Second place MountainCar 2008
1st First place Helicopter Hovering 2009
2nd Second place PolyAthlon 2009

Neural Networks and other utitlies:

python numpy ndlinspace, the n-dimensional linspace function:

Python Code of the n-dimensional linspace function nd-linspace (ndlinspace.py) (python + numpy)

Simple and very useful Multilayer Perceptrons Neural Networks with BackPropagation training:

Python Code (bpnn.py) (pure python)

Python Code (Xbpnn.PY) (python with numeric - fast for big networks)

Java Code (bpnn.java) (pure java implementation)

Matlab simple and nice multilayer perceptron (MLP) with back-propagation training (Matlab BackPropagation) (pure Maltab/Octave implementation). This implementation is specially designed for neuro-evolution since all the weights are represented in a vector which is then automatically decoded in the evaluate function.

Simple and very useful Kohonen's style Self Organized Maps SOM Neural Networks:

Pure python Self Organized Map SOM SOM.py
Pure python Fuzzy Gaussian Kernel Self Organized Map OSOM OSOM.py (you must install Matplotlib only to see the example)

An Evolutionary Neural Network (Multilayer perceptron) with Differential Evolution Training Algorithm:

Python Code (DEBPNN.py) (pure python), please download the file (DESolver.py) which is needed by the Neural Net.
This Network performs better than backpropagation, also it is implemented saving and reading weights to/from file, Enjoy it.

Reinforcement Learning:

A Reinforcement Learning Environment in Python: (QLearning and SARSA) Version 1.0

Download the Package RLearning for python :                       ReinforcementLearning.zip

Also, a win32 installer is provided:                                           RLearning-1.0.0.win32.exe

It includes as examples a Mountain Car Problem and Cart Pole Control Problem:
Finally I have decided to make the graphics output with vpython which I think it is the best option to produce nice and very easy to program real time graphics on python for this kind of things. After the installation, check the examples folder to see the Mountain Car demo and the Cart Pole demo.

Mountain Car Cart Pole

A Reinforcement Learning Environment in Matlab: (QLearning and SARSA)

Please note that in some versions of Matlab you should delete some empty parenthesis in order avoid some errors.
This should not happen with Matlab release from version 7.
This code is a simple implementation of the SARSA Reinforcement Learning algorithm without eligibility traces, but you caneasily extend it and add more features due to the simplicity and modularity of this implementation. Enjoy it!

Matlab SARSA implementation of the Mountain Car Problem:                    SARSA Mountain Car.zip

Matlab SARSA implementation of the Cart Pole Control Problem:                  SARSA CartPole.zip

Matlab SARSA implementation of the Acrobot Control Problem:                SARSA Acrobot.zip

Matlab Multi-Agent SARSA implementation of a Three-Link Planar Robot: SARSA Arm.zip

                           (see the paper for more information on distributed Reinforcement Learning)

Some pictures of the Matlab implementations:

Mountain Car: Cart Pole:

Three-link Planar Robot:
Acrobot:

* Eligibility traces:

Matlab SARSA(λ) (sarsa-lambda) eligibility traces implementation for the Mountain Car Problem: SARSA lambda (eligibility traces) Mountain Car

* Partially Observable Markov Decision Processes:
A collection of experiments to prove the hypothesis of the reduction of POMDPs to MDP by means of the use of Internal Clocks.
I am sorry for not having at this time more theoretical material at hand but you can write me if you want to talk about or even better, join the rl-list at google.
Matlab SARSA Iclock (sarsa plus internal clock) solution to Altered POMDP Mountain Car Problem: SARSA ICLOCK Mountain Car( Altered POMDP Mountain Car Problem. The Agent cannot see the position of the car but only its speed !!!. )

Mountain Car internal clock experiment:

We will extend the list of solved problems frequently. Any comments or suggestions are welcome. Classifier System (XCS) in python: XCSPython.zip 1.0

This is a translation of Martin Butz XCSJava with some pythonic modifications.

Also, I am planning to port some parts tThis is a translation of Martin Butz XCSJava with some pythonic modifications.

Also, I am planning to port some parts to C python extension modules in order to speed up the tool using pyrex.

Please, report any bug.

I am working to free this implementation of any possible bug, despite that its father XCSJava seems to be bug free, this is a translation which may contain unpredictable bugs. When more tests be made I will report that this implementation is bug free.

Cheers.

A collection of neuro-evolution experiments in robotics (CMA-ES and NEAT):

Matlab implementation of neuro-evolution learning for robot control:	EvoNeuroControl.zip

Planar Robot	SCARA Robot

This software is part of a research paper on neuro-evolutionary methods for multi-link robots, such as the three link planar robot and the SCARA robot.

Comments:

Revised: 09/28/09.

Michigan Car Accident Lawyer

Neural Networks and other utitlies:

Reinforcement Learning:

A collection of neuro-evolution experiments in robotics (CMA-ES and NEAT):