ERROR: Data Binding annotation processor version needs to match the Android Gradle Plugin version.
You can remove the kapt dependency androidx.databinding:databinding-compiler:1.0.0 and Android Gradle
Plugin will inject the right version.
This blog will be looking at how to set up and start a Hadoop server on windows as well as give some explanation as to what it is used for.
What is Hadoop:
Hadoop is a set of tools that can be used for easy processing and analyzing Big Data for a company and research. Hadoop gives you tools to manage, query, and share large amounts of data with people who are dispersed over a large geographical location. This means that teams in Tokyo can easily work with teams in New York, well not accounting for sleeping preferences. Hadoop gives a huge advantage over a traditional storage system, not only in the total amount of storage possible, but in flexibility, scalability, and speed of access to this data.
Modules
Hadoop is split up into 4 distinctive modules. Each module performs a certain task that is needed for the distributed system to function properly. A distributed system is a computer system that has its components separated over a network of different computers. This can be both data and processing power. The actions of the computers are coordinated by messages that are passed back and forth between each other. These systems are complex to set up and maintain but offer a very powerful network to process large amounts of data or run very expensive jobs quickly and efficiently.
The first module is the Distributed Filesystem. The HDFS allows files to be stored, processed, shared and managed across a set of connected storage devices. HDFS is not like a regular operating file system and normally can be accessed by any supported OS which gives a great deal of freedom.
The second module is MapReduce. There are two main functions that this module performs. Mapping is the act of reading in the data (or gathering it form each node). Mapping then puts all this data into a format that can be used for analysis. Reduce can be considered the place where all the logic is performed on the collected data. In other words, Mapping gets the data, Reducing analyzes it.
Hadoop common is the third module. This module consists of a set of Java tools that each OS needs to access and read the data that is stored in the HDFS.
The final module is YARN which is the system management that manages the storing of the data and running of task/analysis of the data.
Little More Detail:
What is a namenode? A namenode stores all the metadata of all the files in the HDFS. This includes permissions, names, and block locations. These blocks can be mapped to each datanode. The namenode is also responsible for managing the datanode, i.e. where it is saved, which blocks are on which node, etc…
A datanode, aka a slave node, is the node that actually stores and retrieves blocks of information requested by the namenode.
Installation:
Now with the background out of the way lets try to install the system on Windows 10.
Step 1: Download Hadoop Binaries from here Apache Download Mirrors
Step 2: Make its own folder in the C drive to keep things tidy and make sure that it is easy to find.
NOTE DO NOT PUT ANY SPACES AS IT CAN CAUSE SOME VARIABLES TO IMPROPERLY EXPAND.
Step 3: Unpack the tar.gz file (I suggest 7 zip as it works on windows and is free)
Step 4: To run it on Windows, you need a windows compatible binary from this repo https://github.com/ParixitOdedara/Hadoop. You can just download the bin folder and copy all the files from the downloaded bin to Hadoop's bin (replace any files if needed). Simple right.
Step 5: Create a folder called data and, in this folder, create two others called datanode and namenode. The datanode will hold all the data that is assigned to it. The namenode is the master node which holds the metadata for the datanode (i.e. which data node the 64mb blocks is located on)
Step 6: Set up Hadoop Environment variables like so:
HADOOP_HOME=”C:\BigData\hadoop-2.9.1\bin”
JAVA_HOME=<Root of your JDK installation>”
And add it to your path variables like this
Step 7: Editing several configuration files.
First up is the:
Ect\hadoop\hadoop-env.cmd
set HADOOP_PREFIX=%HADOOP_HOME%
set HADOOP_CONF_DIR=%HADOOP_PREFIX%\etc\hadoop
set YARN_CONF_DIR=%HADOOP_CONF_DIR%
set PATH=%PATH%;%HADOOP_PREFIX%\bin
On the first time you start up you need to run I a cmd write:
Hadoop namenode -format
This set up your namenode and gets Hadoop running.
Now cd into your sbin folder and type
start-all.cmd
This will open up 4 other screens like this
Their names are; namenode, datanode, nodemanager, and resourcemanager.
And now we can look at Hadoop in a browser
The resource manager is http://localhost:8088/cluster/cluster
This is what you should be greeted with
And that’s it, we managed to install and use Hadoop. Now this is a very simple way of doing it and there may be better approaches like using Dockers, or commercial versions which are much easier to use and setup, but learning how to set it up and run it from scratch is a good experience.
Conclusion
We learned that it was very complex to set up and configure all of Hadoop. But with all the power it can bring to Big Data analysis as well as large data sets that are used for AI training and testing, Hadoop can be a very powerful tool in any researcher, data scientist, and business intelligence analyst.
A potential use of Hadoop form image analysis where you have images that are stored in different sources or if you want to use a standard set of images, but the number of images is too large to store locally in a traditional storage solution. Using Hadoop, one can establish a feeding reducer that can then be used in a data generator method in a Keras model. The potentially one can have an endless stream of data from an extremely large dataset, thus giving almost unlimited data. This approach can also be used to get numerical data that is stored on a Hadoop system. Now you do not have to download the data directly just use Hadoop to query and do preprocessing of the data before you feed it into your model. This can save time and energy when working with distributed systems.
Why am I making this post?
Well, I basically needed to make my own SIFT algorithm as there is no free one in OpenCV anymore, well at least 3.0+.
For computer vision, one of the most basic ideas is to extract information from an image. This is feature extraction. There are different levels of features mainly global and local features. This blog will look at SIFT which is a local feature extractor. This is done by finding key points or areas of great change then adds quantitative information or descriptors that can then be used in a more complex task like object detection. Ideally, these key points should be able to be uniquely identified in various images regardless of transformations or changes in the image.
Why Python?
Yes, it's not the best in speed for this, and after running the code it takes a hot minute for it to do the feature extraction. But, I can easily use it in any computer vision project that I have now and it plugs and play no problem.
How does SIFT work?
First, you give SIFT a picture to work with, we will be using an image I took of a dog from when I went dog sledding in Finland.
Step1: Double the size of your image both using bilinear interpolation.
Step 2: Blur the image using Gaussian Convolution.
Step 3: Preform more convolutions using Standard Deviation.
Step 4: Downsample each image.
Step 5: Restart the convolution again.
Continue this until the image is too small to perform these steps anymore.
This is called a scale-space which will help simulate many different scales that an image can come in (i.e. from small to larger and everything in between).
After the convolution, we will have to get the Laplacian for each scale space. This gives a grey scale value for each element in the image. The max values for the Laplacian will then be our key points. The max pixel will be a pixel whose value is larger than all its surrounding other pixels. This can be extended to several pixels or even a larger area depending on your needs. We can refine the key point results in the scale space by using a Quadratic Taylor expansion. Along with this, the identification of key points that lie on the edge of an object should also be removed as they are poor key points as they are not unique to translations of an image and are parallel to the edge direction. The point of keypoint extractions is not to find an edge of an object, but rather to find unique features of the image which may or may not lie on the edge of a target. That is the difference between edge detection and key point detection.
Finally, SIFT will give a reference orientation to each key point. Then the gradient of the image will be calculated by finite differences. Then smooth the gradients of the image using box blurs. This will allow for the remaining points that exceed a certain value/threshold to be kept. Another key point will be discarded.
After all of that, we have a list of final key points that we can create the descriptors for. This is done by creating a histogram of the gradient directions for each key point. This will not just make one histogram as it will make several around in a circle of the pixel where the histogram corresponds to the center pixel. Each gradient is a circle shape (rather than a box as used previously), and any key point that cannot create a full circle will be discarded.
So, after all of this, we now are left with a set of key points that are local features that can help us identify unique objects in images. This can then be used on its own for simple computer vision tasks like object identification, image slicing, and binding, or even image similarity for search engines. So, SIFT can be very useful if you know how and why it works.
Code for SIFT
Here is the pseudo code for getting SIFT for key point detection.
Sift Code
Find_Key_Points(image):
Gaussian Smoothing (image)
Downsample the image
Make Gaussians Pyramids
Create Downsample Gaussians Pyramids
For each octave:
Start Extrema detection.
for each image sample at each scale:
find the gradient magnitude
find the orientation
Calculate each keypoints orientation
Calculate each keypoints descriptor
As you can see the key points found in this image is not perfect, but the implementation of SIFT is not very easy. Also, this image has a lot of noise in it so it may look like the algorithm did not work, but it is working fine, but the changes are on such a small level that it is difficult to even see it.
And for those who want to see here is the whole (all be it abridged) code
Now for a more real-world example. Getting data from the Market1501 data set that holds several images of people for different tasks. By running one of the images through the key point extractor, it allows for you to find the local features that are unique to the image itself.From the above picture you can see that a few of the key points are generated that look great, and one that is over on the street which is what you do not want. This is not the SIFT extractors fault it is looking over the whole image and not just the person. If you want a better result for a person's individual key points without the background mess, I would suggest using segmentation to create a mask and then use the results in the SIFT extractor. That way it will limit what is being looked at and will give better local features for each person.
Conclusion
This implementation is not the state of the art method for getting key points and is very susceptible to blurry or poor image quality, as well as background noise and noise in the image that you cannot detect yourself. This took a long while to implement and the results are not that great compared to the time it took to create this. In my opinion, I would look into other means of extracting local features like the segmentation of the image.
Reference
Otero, I. R., & Delbracio, M. (2014). Anatomy of the SIFT Method. Image Processing On Line,4, 370-396. doi:10.5201/ipol.2014.82
Google Cloud Vision API
言わずと知れた Google さんの出しているAPIです。
一度画像をアップロードする必要があるものの、
様々なファイルタイプに対応しており、言語も自動で判別してくれます。
1ヶ月に1000ユニットまでは無料ですが、それを越えると1000ユニットごとにお金がかかります。
Q -learning is a model-free approach to reinforcement learning. This means that Q-learning does not rely on a model to make decisions based on an input, but instead uses a policy that shows the agent what actions to perform given a certain circumstance. Q-learning should find the best policy which gives an optimal solution to any finite Markov decision process. The FMDP can be described as:
Where X is the state space, A is the action space, and r is the reward function. The optimal solution is obtained by maximizing the total reward for any action and any change in state, starting with the current state. The best or optimal solution can be found for any finite problem given an unlimited exploration time combined with an exploration adaption to a policy. This optimal Q-function can be calculated as such:
The full equation and explanation can be found in Melo’s Convergence of Q-Learning: A Simple Proof [1]:
Influences on how the agent will learn using Q-learning comes down to three real factors. Learning rate controls how much, or the rate in which new information replaces older information. If the learning rate is set to 0, then the agent will never learn anything new and will rely on previous knowledge to choose its actions. If the rate is set to 1, then the agent will only look at the most recent data and completely discard previous data.
The Discount factor is another variable that controls learning. The discount factor determines the weight of future rewards and how important or not these rewards will be. If it is set to 0, then the model will only consider instant rewards or the current reward and not look towards potential rewards of future actions. If the discount factor is set to 1, then the agent will work towards achieving the long-term highest reward.
The third and final variable used for learning the control is the initial conditions or Q0. Q-learning is an iterative program, it must have a starting condition before any changes take place to any state or environment. To help an agent explore options better, high initial values should be used regardless of which action is used. The update rules can control the values compared to other alternatives which can increase its total probability of being chosen. This will allow for each option to be fully explored and to aid in finding an optimal solution to each problem.
Q-Tables:
Q-Learning runs off a table or matrix of state and actions. This Q-table starts out all values as 0 and after each episode, these values are updated accordingly. The matrix should take the size of both the state as well as the action size with a very basic form of this is a simple array [state, action]. The simplest method for creating a Q-table in python is this:
This Q-table will store every action and state that the agent will be in. Exploitation is done when an action that has been performed before and in a certain state is chosen which results in a reward (most likely the max possible future reward). Exploration is choosing a random action at any point in time and in any state. A basic method for this could be defined as below:
import random
epli = 0.4defchoose_action (epli)::
ran_float = random.random() # Generates a random float between [0,1]if ran_float < epli:
#preform a random actionelse:
get_action_from_qtable(current_state, q_table)
After choosing an action to perform and receiving a reward, the Q-table must be updated. The updating must be done after each action is performed. The updating phase will end at the conclusion of each episode. The basic flow of the update function will be:
Step 1: Get the current state of the agent.
Step 2: The agent takes action. (Picks random action or Pick action from Q-table)
Step 3: Update q-value.
An update rule for Q-values can be described as:
This is a basic approach to adjust and update q-values in a Q-table.
Now for a simple example:
First lest look at the agent in the game:
classAgent:
def__init__(self, learning_rate = 0.1,
discount = 0.95,
exploration_rate = 1.0,
iterations = 1000):
# Q talbe for holding rewards
self.q_table = [[0,0,0,0,0], [0,0,0,0,0]]
self.learning_rate = learning_rate
self.discount = discount # Future rewards value to current rewards
self.exploration_rate = exploration_rate # Exploration rate# This controls sift from exploration to exploation
self.exploration_delta = 1.0 / iterations
# Chose between explotation or explorationdefget_next_action(self, state):
if random.random() > self.exploration_rate:
return self.greedy_action(state)
else:
return self.random_action()
defgreedy_action(self, state):
# Check to see if forward is best rewardif self.q_table[FORWARD][state] > self.q_table[BACKWARD][state]:
return FORWARD
elif self.q_table[BACKWARD][state] > self.q_table[FORWARD][state]:
return BACKWARD
# Rewards are equal, take random actionreturn FORWARD if random.random() < 0.5else BACKWARD
defrandom_action(self):
return FORWARD if random.random() < 0.5else BACKWARD
defupdate(self, old_state, new_state, action, reward):
old_value = self.q_table[action][old_state]
# best next action
future_action = self.greedy_action(new_state)
# reward for the best next action
future_reward = self.q_table[future_action][new_state]
# Main Q-table updating algorithm
new_value = old_value + self.learning_rate * (reward + self.discount * future_reward - old_value)
self.q_table[action][old_state] = new_value
# Finally shift our exploration_rate toward zero (less gambling)if self.exploration_rate > 0:
self.exploration_rate -= self.exploration_delta
Then lets built the environment that he can walk through.
# Build the EnvironmentclassEnvironmentSimulator:
def__init__(self, length=5, small=2, large=10):
self.length = length # Length of the environment
self.small = small # reward for going back to the start
self.large = large # reward for reaching the end
self.state = 0# environment entry pointdeftake_action(self, action):
if action == BACKWARD:
reward = self.small
self.state = 0elif action == FORWARD:
if self.state < self.length - 1:
self.state += 1
reward = 0else:
reward = self.large
return self.state, reward
# Reset the environmentdefreset(self):
self.state = 0return self.state
This is a very simple array of 5 spaces. So basically, the agent can walk back a forth and get a reward for either reaching one end or the other.
Next, we will set up the environment and the main loop for running the simulation.
# Set up environment
environment = EnvironmentSimulator()
environment.reset()
# Scores
total_reward = 0
last_total = 0for step inrange(iterations):
# Store the current state in old state
old_state = environment.state
action = agent.get_next_action(old_state)
new_state, reward = environment.take_action(action)
agent.update(old_state, new_state, action, reward)
total_reward += reward
if step % 250 ==0:
performance = (total_reward - last_total) / 250.0print("Step:{} Performance:{} Total-reward:{}".format(step, performance, total_reward))
RI Hello World Revisited
Let's take a look back at the Cart Pole example from the previous blog, tech.unifa-e.com
We did that with a Keras model, now we will do it with a q-table approach.
Next, we need a few methods for choosing actions and updating the tables:
# Q learning functiondeflearnQ(self, state, action, reward, value):
old_value = self.q.get((state, action), None)
# If there are no values in the table add the rewardif old_value isNone:
self.q[(state, action)] = reward
else:
self.q[(state, action)] = old_value + self.alpha * (value - old_value)
defchooseAction(self, state):
q = [self.getQ(state, a) for a in self.actions]
maxQ = max(q)
if random.random() < self.epsilon:
minQ = min(q); mag = max(abs(minQ), abs(maxQ))
q = [q[i] + random.random() * mag - .5 * mag for i inrange(len(self.actions))]
maxQ = max(q)
count = q.count(maxQ)
if count > 1:
best = [i for i inrange(len(self.actions)) if q[i] == maxQ]
i = random.choice(best)
else:
i = q.index(maxQ)
action = self.actions[i]
return action, q
deflearn(self, s1, action, reward, s2):
maxqnew = max([self.getQ(s2, a) for a in self.actions])
self.learnQ(s1, action, reward, reward + self.gamma*maxqnew)
And now let's define the main method:
if __name__ == '__main__':
# Load the cart pole game from gym ai
env = gym.make('CartPole-v0')
# Vriables needed
max_number_of_steps = 200
last_time_steps = numpy.ndarray(0)
n_bins = 8
n_bins_angle = 10
max_number_episodes = 200
episode_number = 0
render = False
number_of_features = env.observation_space.shape[0]
last_time_steps = numpy.ndarray(0)
# Simplfy the number of states as it is too large
cart_position = pandas.cut([-2.4, 2.4], bins=n_bins, retbins=True)[1][1:-1]
pole_angle = pandas.cut([-2, 2], bins=n_bins_angle, retbins=True)[1][1:-1]
cart_velocity = pandas.cut([-1, 1], bins=n_bins, retbins=True)[1][1:-1]
angle_rate = pandas.cut([-3.5, 3.5], bins=n_bins_angle, retbins=True)[1][1:-1]
qlearn = QLearn(actions=range(env.action_space.n),
alpha=0.5, gamma=0.90, epsilon=0.1)
while episode_number < max_number_episodes:
observation = env.reset()
cart_position, pole_angle, cart_velocity, angle_rate_of_change = observation
state = build_state([to_bin(cart_position, cart_position),
to_bin(pole_angle, pole_angle),
to_bin(cart_velocity, cart_velocity),
to_bin(angle_rate_of_change, angle_rate)])
for t inrange(0, max_number_of_steps):
if render:
env.render()
action = qlearn.chooseAction(state)
observation, reward, done, info = env.step(action)
cart_position, pole_angle, cart_velocity, angle_rate_of_change = observation
nextState = build_state([to_bin(cart_position, cart_position),
to_bin(pole_angle, pole_angle),
to_bin(cart_velocity, cart_velocity),
to_bin(angle_rate_of_change, angle_rate)])
ifnot(done):
qlearn.learn(state, action, reward, nextState)
state = nextState
else:
# Penalize it for failing
reward = -200
qlearn.learn(state, action, reward, nextState)
last_time_steps = numpy.append(last_time_steps, [int(t + 1)])
break
And that is really the only difference between the Keras and q-learning ways of solving the Cart Pole problem.
Instead of the model deciding what to do, the values in the q-table are making the decision.
Conclusion
This shows how to simply set up an environment and use a q-table to learn RI. This was a simple example the next logical step would be to have a negative reward which would encourage the AI to stop moving backward and to only move forward. This could help shape how the AI will learn and help move it towards the desired actions. This program is really a bare-bones approach with no bells and whistles. It shows you what is the minimum to have for an agent, environment, and simulation for training up a RI agent.
We also revisited the cart pole example to see how Q-Learning can be done compared to using a Keras model. The Keras model needs less code and can solve the issue a little better than the Q-learning approach. This may be because I had to simplify the states that the pole can take which may lead to the poorer results.
Q-learning has its limits as the table can become very complex as the states and actions that are a possible increase in the total number. This can greatly increase the complexity of the Q-table which can become very difficult to manage. By simplifying the dimensions of the state/action sets can help make it more manageable, but you sacrifice granularity which can lead to mistakes in the results.
Resources:
[1]: Melo, F. Convergence of Q-learning: a simple proof. Retrieved 2019 from