Saturday, 16 February 2019

My experience in Diverse Intelligence Summer Institute, University of St Andrews, Scotland

Last summer (September 2018), I participated in Diverse Intelligence Summer Institute, University of St Andrews, Scotland. The goal of this DISI is to gather scholars and students working in the field of cognition and build connections. The "intelligence" includes the human brain, animal cognition, artificial intelligence and even plant intelligence. The summer school was intensive two weeks of lectures, discussions, a project and for sure fun.
During those two weeks I met scholars from my field of artificial intelligence and had new friendships with smart students from different countries. This experience opened my mind to other research work that I haven't thought about. Human and animal cognition for example is highly related to develop a machine cognition. Before this summer school, my focus was mainly on the computational side of the artificial intelligence. During and after the summer school my imagination got broader and new perspective where opened in my mind. Besides, I learned about cultures and of nations from other part of the world where I couldn't visit such as the US.
I would encourage every student to take the opportunity to participate in summer school activities as it brings a knowledge and experiences that are not found in the daily university life or even in scientific conferences. Although it was a dense course, I extremely enjoyed it with the kind organizers.

Sunday, 10 February 2019

Part 3: Artificial cognitive systems, Reinforcement Learning

In Part 1 we discussed the logical approach of cognitive systems and how can we model an AI agent by giving it the logical rules and the knowledge to deduct the output. Next, to let the agent learns by itself, we presented machine learning approach that can find correlation between input data set and their given output, as in Part 2.
For an interactive system, it is difficult to give all the data set of the possible experiences. The agent needs to try and collect experiences by itself. The approach that allows the agent to try and fail to learn from its experiences is the reinforcement learning. Deep Reinforcement Learning combines both the supervised approach of finding the correlation between input and output data set and the trial and error approach. In DRL, the input includes the state that the agent collected by experiences. The output could either be the action to take or an information that can help selecting the action.
One method is Deep Q-Learning. This method calculates the Q-value for each action and then selects the action that yields the highest Q-value. This approach uses the Bellman equation for calculating the Q-value and the neural network to find the correlation.
Another approach uses deep learning for generating the policy using the policy gradient. This method is used for continuous action-space.
We modeled a deep Q-learning agent to drive in a physics game simulator OpenDS. The agent receives a scenario message that describes the state of the car in the road. The agent selects the action in a maneuver message to control the driving. For a lane keeping task, the state reflects the distance from the side of the road, the lane heading as an angel between the car heading and the road, the curvature of the road and the steering angel.

Sunday, 3 February 2019

Part 2: Artificial cognitive systems, Machine Learning

In Part 1, we discussed the logical approach of the artificial cognitive systems. We mentioned that it is based on rules and knowledge. These rules and knowledge are pre-programmed in the system. This makes the system static and doesn't learn.
Another approach is the data-driven system, which is called machine learning. In this approach, the system gets a data set with real examples of the input and the output and the system tries to find the correlation in the data based on statistical calculations. A famous example, is a data set of house prices. In this example, each house is identified with features such as house area, number of rooms, etc. and the system tries to predict the best price for the houses. The system finds the correlation between the house features and its prices without explicitly giving it a rule or knowledge. Such a system that can adjust itself by itself based on examples, is considered a learning system.
Deep learning is a recent branch of machine learning. Deep learning is a multi-layer neural network that finds the correlation between the input and the output in a gradient descent mechanism. Deep learning has been widely used in more complex data sets such as text, images, audio and video. It went beyond classification and prediction. It has been used for interactive systems such as autonomous cars. In the autonomous car system, the driver collects sensory data such as cameras along with the action control such as the steering. After a long driving, a big driving data is collected. The data is then fed to the network to train it to predict the best action given a sensory data. After that, by deploying the network into the car with autonomous mode, the network is able to drive by itself is a similar road.
An excellent example is a tutorial by Siraj Raval about self-driving cars. Try it and enjoy!