ai-maker.atrilla.netA.I. Maker | A Maker’s Approach to Artificial Intelligence

Skip to content A Maker’s Approach to Artificial Intelligence Primary Menu Home About GitHub RSS Contact World Radio Day… and my RTL-SDR dongle! February 13, 2016 · atrilla It’s been a while folks, but hey, here I am again, celebrating the World Radio Day with a brand new RTL-SDR dongle! If you had never heard about software-defined radio (SDR) before, it’s probably because it’s a relatively new field for hobbyists, four years at the most (compared to the first MIT computer hacker groups of the seventies). Prior to that, the complex mathematical computations that are required to deal with radio systems had to be implemented in hardware, which made them very expensive. In 2012, though, a group of “wizards” (Eric Fry, Antti Palosaari and the Osmocom team) discovered that the Realtek chip (RTL2832U) that some DVB-T dongles incorporate could be exploited so as to provide access to the raw signal data, which enabled it to be converted into a computer-based wideband radio scanner. Its features do not equal those of a dedicated piece of SDR electronics, but it’s great for having some fun. With this RTL-SDR dongle you can tune into FM Radio, AM signals (garage door remotes), CW (morse code), unencrypted conversations (such as those used by many police and fire departments), POCSAG pagers, satellites, the ISS, etc. RTL-SDR dongle The two main radio components of the RTL-SDR dongle are the tuner (R820T2), note how Nooelec advertises it on the plastic cover, which downconverts the modulated-carrier signal into baseband (grossly speaking, this can also be an intermediate frequency), and the high-speed Analog-to-Digital converter (RTL2832U) that samples it and makes it ready for further digital signal processing. This device can deal with frequencies between 24MHz and 1766MHz, with a bandwidth around 3MHz and 8 bits per sample. Further details here . In future posts I will delve into the nature of signals, how they are represented, modulated… and the frequency up/down conversion processes to establish the wireless communication. Stay tuned! Posted in Radio Leave a comment Artificial Neural Networks, to the point May 17, 2015 August 24, 2015 · atrilla Artificial Neural Networks are powerful models in Artificial Intelligence and Machine Learning because they are suitable for many scenarios. Their elementary working form is direct and simple. However, the devil is in the details, and these models are particularly in need of much empirical expertise to get tuned adequately so as to succeed in solving the problems at hand. This post intends to unravel these adaptation tricks in plain words, concisely, and with a pragmatic style. If you are a practitioner focused on the value-added aspects of your business and need to have a clear picture of the overall behaviour of neural nets, keep reading. Note that the neural network is plausibly renown to be the universal learning system. Without loss of generality, the text below makes some decisions regarding the model shape/topology, the training method, and the like. These design choices, though, are easily tweaked so that the same implementation may be suitable to solve all kinds of problems. This is accomplished by first breaking down its complexity, and then by depicting a procedure to tackle problems systematically in order to quickly detect model flaws and fix them as soon as possible. Let’s say that the gist of this process is to achieve a “lean adaptation” procedure for neural networks. Theory of Operation Artificial Neural Networks (ANNs) are interesting models in Artificial Intelligence and Machine Learning because they are powerful enough to succeed at solving many different problems. Historical evidence of their importance can be found as most leading technical books dedicate many pages to cover them comprehensibly. Overall, ANNs are general-purpose universal learners driven by data. They conform to the connectionist learning approach, which is based on an interconnected network of simple units. Such simple units, aka neurons, compute a nonlinear function over the weighted sum of their inputs. You will see this clearly with the equations below. Neural networks are expressive enough to fit to any dataset at hand, and yet they are flexible enough to generalise their performance to new unseen data. It is true, though, that neural networks are fraught with experimental details and experience makes the difference between a successful model and a skewed one. The following sections cover the essentials of their modus operandi without getting bogged down in the small details. Framework Some say that 9 out of 10 people who use neural networks apply a multilayer perceptron (MLP). A MLP is basically a feed-forward network with 3 layers (at least): an input layer, an output layer, and a hidden layer in between, see Figure 1. Thus, the MLP has no structural loops: information always flows from left (input)to right (output). The lack of inherent feedback saves a lot of headaches. Its analysis is totally straightforward given that the output of the network is always a function of the input, it does not depend on any former state of the model or previous input. Figure 1. Framework of a multilayer perceptron. Its behaviour is defined by the weight of its connections, which is given by the values of its parameters, i.e., the thetas. Framework of a multilayer perceptron. Regarding the topology of a MLP it is normally assumed to be a densely-meshed one-to-many link model between the layers. This is mathematically represented by two matrices of parameters named “the thetas”. In any case, if a certain connection is of little relevance with respect to the observable training data, the network will automatically pay little attention to its contribution and assign it a low weight close to zero. Prediction The evaluation of the output of a neural network, i.e., its prediction, given an input vector of data is a matter of matrix multiplication. To that end, the following variables are described for convenience: is the dimension of the input layer. is the dimension of the hidden layer. is the dimension of the output layer. is the dimension of the corpus (number of examples). Given the variables above, the parameters of the network, i.e., the thetas matrices, are defined as follows: The following sections describe the ordered steps that need to be followed in order to evaluate the network prediction. Input Feature Expansion The first step to attain a successful operation of the neural network is to add a bias term to the input feature space (mapped to the input layer): The feature expansion of the input space with the bias term increases the learning effectiveness of the model because it adds a degree of freedom to the adaptation process. Note that directly represents the activation values of the input layer. Thus, the input layer is linear with the input vector (it is defined by a linear activation function). Transit to the Hidden Layer Once the activations (outputs) of the input layer are determined, their values flow into the hidden layer through the weights defined in : Similarly, the dimensionality of the hidden layer is expanded with a bias term to increase its learning effectiveness: Here, a new function is introduced. This is the generic activation function of a neuron, and generally it is non-linear, see below. Its application yields the output values of the hidden layer and provides the true learning power to the neural model. Output Prediction Finally, the activation values of the output layer, i.e., the network prediction, are calculated as follows: Activation Function The activation function of the neuron is a non-linear function that provides the expressive power to the neural network. Typically, the sigmoid function or the hyperbolic tangent function is used. It is recommended this function be smooth, differentiable and monotonically non-decreasing (for learning purposes). Note that the range of these functions varies from to ,...