The Brain Project - Vision

Vision - The breakdown of one of our 5 senses
Color

One of the senses that humans are graced with is vision. Vision comes from the use of our eyes and how the brain processes the data it receives. There are two eyes that allow humans to see things in 3-dimensions. The AI's vision would work similarly, using two very specialized cameras that are programmed to see what a human sees to the best ability of the technology available. Ultimately, every thing that a human looks at can be broken down into ideas of shapes and colors, which can be translated into numerical data and therefore processed effectively by the AI's "brain." To illustrate the points I have made, I'm going to begin with posting a simple picture, theoretically a picture "seen" by the AI, and describe in detail how the AI's "thought process" effectively analyzes and learns unique data from a vision standpoint.

an example of an environment seen by the AI
and the processing of the data that environment provides

Vision is the first sense I am going to describe. Keep in mind that I have not worked in data from other senses, such as sound, taste, and feel. These functions will be added and discussed later, I am only dealing with vision in this entry.

The AI would need several processes programmed in to analyze the data that is provided with this picture. It must have a sense of measurement (length, width, height) as it relates to the size of the AI and other objects around it, as well as a sense of color, and the logical processes that allows the future utilization of the data it has received just by using its "vision."

Color can be quantified, and the most common way is through the RGB and/or hex color systems. I am more familiar with the hex color system, so I will be using that for this example. (You can see a list of major hex color codes here.) The AI would be able to pick up on the colors in what it is seeing by referring to their hex designation for whatever color it is seeing. The color of objects can provide a lot of valuable data. When this color data is combined with the data surrounding the shapes (and measurements) of objects, a fuller, more diverse idea of the environment is able to develop with nothing else but the AI's camera eyes data.

So, I'm going to begin to analyze the picture I have posted based on quantifiable data, allowing the processing of that data in the "brain" such that it can learn from it and derive its own opinions and experiences. So, to start simply, we know that the picture is made up of specific colors and shapes. These colors and shapes can be analyzed, and I will illustrate that.

For each color in the picture, there is a hex (or RGB) code that defines that color in a numerical fashion. The AI would be able to notice and store the data that corresponds to the colors it is seeing, defining it as a hex code. Every color the AI can see is recorded and stored. Each color is stored both as a situational specific color (color in reference to the objects in their room and their eventual function) as well as a way to process and store the same data as it connects with other experiences and situations. Basically, every hex code comes with its own storage system specific to that code, which also includes connections to similar colors (as commonly seen with another color) and the environment in which they appear, based on several factors relating to past experiences.

So, using the picture I provided above as an example, the AI would notice that some of the colors are varying type of brown, it would also note the amount of shade added to each color when it appears (by adding either white or black) so that it may use this data to later understand shadow and highlight. The AI would make note of the layout and arrangement of colors and store that information to eventually distinguish what is a singular object, and what is not, using its observations of shape along with color.

NEXT: Shapes.