“Inertial measurement units (IMUs) are widely used to continuously and steadily provide multi-axis position information from accelerometers, gyroscopes, and other sensors. Due to the multiple degrees of freedom (DOF) nature, all generated data (i.e. the combined data stream from these devices) keeps the system processor in awake mode, and since the raw IMU data is sifted to extract useful gesture and system location information, overwhelm them. Designers need to find ways to offload the main processor so that it doesn’t take on such filtering functions. Machine learning may solve this problem.
“
Author: Steve Leibson
Inertial measurement units (IMUs) are widely used to continuously and steadily provide multi-axis position information from accelerometers, gyroscopes, and other sensors. Due to the multiple degrees of freedom (DOF) nature, all generated data (i.e. the combined data stream from these devices) keeps the system processor in awake mode, and since the raw IMU data is sifted to extract useful gesture and system location information, overwhelm them. Designers need to find ways to offload the main processor so that it doesn’t take on such filtering functions. Machine learning may solve this problem.
This article will first briefly describe the purpose of the IMU, followed by an introduction to STMicroelectronics’ LSM6DSO. The device then illustrates how adding and integrating machine learning and decision tree processing capabilities in the IMU can offload real-time position and motion processing to the host application processor, and how these capabilities can be used in real-world applications.
Introduction to IMUs
The IMU integrates various motion sensors into one device to provide high-precision positioning information. IMUs can be used in a variety of applications, including consumer (cell phones), medical (imaging), industrial (robotics), and military (heading tracking). The device responds to sensor motion and contains one or more of the following motion sensor types:
Gyroscopic sensors measure changes in angular position, usually expressed in degrees per second. Integrating angular velocity over time measures the angle of travel, which is used to track changes in direction. Gyroscopes track relative motion independent of gravity, so errors from sensor bias or integration cause a positional error called “drift,” but can be compensated for by software.
Accelerometers measure linear acceleration, including components of acceleration due to device motion and acceleration due to gravity. Acceleration is measured in g, 1 g = Earth’s gravity = 9.8 m/s2. Accelerometers are divided into single-axis, dual-axis, and three-axis, which are defined as X, Y, and Z coordinate systems, respectively.
Magnetic sensors measure the strength of the magnetic field, usually in microtesla (µT) or gauss (100 µT = 1 gauss). The most commonly used magnetic sensor in mobile Electronic devices is the three-axis Hall-effect magnetometer. By calculating the detected angle of the Earth’s magnetic field and comparing this measured angle with the gravity measured by the accelerometer, the device’s heading relative to the magnetic north pole can be measured very accurately.
Motion tracking using the IMU employs sensor fusion technology to derive a single, high-accuracy estimate of relative device orientation and position from a known origin and orientation. Sensor fusion is mostly software, using complex mathematical algorithms developed by the IMU manufacturer or application developer to combine the IMU’s various motion sensor outputs.
Using sensor fusion for position calculations results in the following measurements:
• Gravity C Specifically Earth’s gravity, excluding the acceleration caused by motion that is sensed by the device. When the IMU is stationary, the accelerometer measures the gravity vector. When the IMU is in motion, the gravity measurement needs to fuse the accelerometer and gyroscope data and subtract the acceleration caused by the motion.
• Linear acceleration, C, is equal to the device acceleration measured by the accelerometer, but the gravity vector is subtracted by software. IMU linear acceleration can be used to measure motion in three-dimensional space.
• Orientation (altitude) C A collection of Euler angles, including yaw, pitch, and roll, measured in degrees.
• The rotation vector C is derived from a combination of data from the accelerometer, gyroscope, and magnetometer sensors. A rotation vector represents an angle of rotation around a specific axis.
IMU error sources
A gyroscope detects orientation through changes in angular velocity, but it tends to drift over time because it only detects changes without a fixed frame of reference. By adding the accelerometer data to the gyroscope data, the software can minimize the gyroscope bias, resulting in a more accurate position estimate. The accelerometer detects changes in orientation relative to gravity, and this data can be used to orient the gyroscope.
Accelerometers are more accurate for static (as opposed to dynamic) calculations. It is better to use a gyroscope to detect orientation when the system is already in motion. Accelerometers respond quickly, and if only this data is used, accelerometer jitter and noise can accumulate errors. Also, accelerometers tend to distort acceleration due to external forces like gravity, which also accumulates as noise in the system. Filtering this data improves accuracy.
By combining the short-term accuracy of a gyroscope with the long-term accuracy of an accelerometer, relying on the strengths of each sensor to offset or at least mitigate the weaknesses of the other, more accurate orientation readings can be obtained. The complementarity of the two sensor types helps reduce errors, but there are other ways to reduce errors.
Fusion filtering can be used to reduce errors
The IMU software uses filtering to minimize positioning errors in the IMU data. There are multiple filtering methods for fusing sensor data, each with varying degrees of complexity. Complementary filtering combines high-pass gyroscope filtering and low-pass accelerometer filtering. Therefore, high frequency noise in the accelerometer data is filtered out in the short term and smoothed with the gyroscope data.
The computational power required to perform all this sensor processing, filtering, and fusion is energy-intensive; this can be a problem for battery-operated systems, especially if the IMU information is not required to be continuously transmitted. For many embedded applications, power consumption can be significantly reduced if the IMU can generate an interrupt to wake the host processor from sleep mode to initiate processing or take some action as a result of the interrupt. To achieve this functionality, some IMU vendors are beginning to incorporate processing and decision-making functions into the IMU.
Let the IMU think
STMicroelectronics’ 6DOF LSM6DSO is one such IMU. The 6DOF LSM6DSO contains three microelectromechanical systems (MEMS) gyroscopes and three MEMS accelerometers that can detect orientation changes and gestures without supervision or assistance from the host processor, all of which are handled on-board. When operating in maximum performance mode, the IMU consumes 0.55 milliamps (mA).
In this mode, the LSM6DSO can continuously monitor its own altitude and motion in space, and can generate interrupts under predetermined conditions to wake up the host processor to perform additional processing of the sensor stream. Using an always-on, low-power IMU is effective because it puts the host processor to sleep and only wakes up when necessary. This energy saving method is worth trying and trusting in battery powered systems.
In addition to the gyroscope and accelerometer sensors, the LSM6DSO IMU contains a signal conditioning and filter block, a finite state machine (FSM) that can run up to 16 programs (all programs share a configurable common output data rate), and a Machine Learning Kernel. Using these resources together, event detection interrupts can be generated when:
• Free fall
• wake up
• 6DOF orientation
• Single and double click detection
• Active/Inactive Identification
• Still/Motion Detection
The signal conditioning block scales the raw sensor data using conversion factors stored in the sensitivity registers. Then, the raw IMU sensor data stream is converted to a 16-bit half-precision floating point (HFP) format so that it can be understood by the FSM. The IMU’s MEMS sensors (accelerometer and gyroscope) along with two analog-to-digital converters (ADCs) and four filter blocks are shown in Figure 1. The filter block is used to convert the MEMS sensor’s analog signal into a filtered digital data stream.
Figure 1: The LSM6DSO IMU uses two ADCs to convert analog signals from the internal MEMS accelerometer and gyroscope to a digital stream. Following the ADC are four digital filters that condition the signal for decision-making by the internal FSM and machine learning cores and the host processor. (Image credit: STMicroelectronics)
The programmable FSM consists of a configuration block and 16 program blocks. The FSM configuration block configures and controls the entire FSM. For the 16 FSM blocks, each block consists of an input selector block and a code block (Figure 2). Both blocks are controlled by values written to the IMU’s internal registers.
Figure 2: The LSM6DSO IMU contains 16 FSM blocks, each of which consists of an input selector block and a code block. (Image credit: STMicroelectronics)
The input selector block sends selected input data from an IMU internal sensor or an external sensor connected to the IMU sensor hub to the code block. The IMU sensor hub can accommodate up to four additional external sensors, such as magnetometers, which can be connected to the IMU through the I2C port.
The FSM code block contains a state machine program. The fixed portion of the program block data section consists of six bytes that define the program’s threshold number, hysteresis, mask, and timer settings. The variable data segment of the program block holds the actual threshold, hysteresis, mask, and timer settings for each program, as defined by the values stored in the fixed portion of the data segment.
The fixed portion of the data segment also defines the size of the variable portion of the code block memory footprint, the programmable reset vector, and the program counter. Since these values are all 8 bits, the maximum number of bytes per FSM program is 256.
The instruction section of the program block contains the actual FSM program. Program instructions include various opcodes for checking sensor inputs against thresholds, checking for zero crossings, and checking timer values for timeouts. The opcode also specifies the conditions required to transition from the current FSM state to the next FSM state. In addition, some instruction opcodes are used to: select thresholds and masks stored in the variable data segment of the block; set the IMU sensor hub multiplexer to connect to one of four possible external sensors; and Interrupt to assert.
Each FSM program can generate an interrupt and modify the corresponding register value according to the selected input signal. These register values are used to transfer data from the IMU to the host processor.
An FSM can be thought of as a microprocessor that lacks an arithmetic logic unit. The FSM can make selections, perform comparisons, and determine the next state based on the results of the comparisons. But FSM can only compute boolean values based on the comparison result.
FSMs are not microprocessors. It can compare and make simple changes to the program flow based on the results of the comparison. FSMs are very simple machines and can therefore be programmed directly with FSM opcodes. FSM does not have a high-level language compiler, but programs are usually so simple that a compiler is not required.
Use FSM
The FSM of the LSM6DSO IMU can be programmed to generate interrupt signals activated by predefined motion modes. The FSM can run up to 16 independent programs simultaneously to detect motion. Each FSM program consists of a series of “if-then-else” steps, taking as input the sensor data streams from the LSM6DSO’s accelerometers and gyroscopes. If any FSM program detects a state that matches the preprogrammed pattern, the FSM generates an interrupt to the host processor.
Of the 16 available FSM programs, each contains three memory segments for fixed data, variable data, and instructions. A single FSM block diagram is shown in Figure 3.
Figure 3: The FSM in STMicroelectronics’ LSM6DSO IMU contains 16 code blocks, each containing three memory segments for fixed data, variable data, and instructions. (Image credit: STMicroelectronics)
The structure of a single program in a code block consists of three sections in a memory block:
• Fixed data segment, which is the same size in all FSM programs
• Variable data segment, which can be of variable size
• Instruction section, containing conditions and commands
When programming each FSM code block, these three memory segments need to be loaded with programmed values that determine the behavior of the FSM. STMicroelectronics provides FSM programming tools in the downloadable Unico evaluation development software and development environment. STMicroelectronics also includes several FSM sample programs with the Unico development tools to help users learn FSM programming. These sample programs demonstrate several IMU-based interrupt scenarios, including:
• Basic pedometer
• Free fall system
• Simple motion detection
• Picked up system
• Swing system
• A system that stops motion (standstill)
• wrist tilt
The FSM sample programs demonstrate the use of various FSM functions. Any of these sample programs can be installed into an IMU demonstration platform such as the STEVAL-MKI109V3 eMoTIon STM32 evaluation board, which has a 28-pin socket that plugs into the LSM6DSO STEVAL-MKI197V1 IMU adapter board. Programming the STEVAL-MKI109V2 board using one of the sample programs is just a few clicks away in the Unico development environment.
However, the LSM6DSO is much more than that.
Machine Learning Core
The LSM6DSO IMU also includes a more sophisticated programmable pattern matching engine called the Machine Learning Core. This enables the identification of various types of motion using multiple sensor data streams from the internal IMU sensor and any connected external sensors. Recognized activity categories include stationary (no movement), walking, jogging, cycling, and driving. Activity classification takes the form of decision trees at the heart of machine learning.
The machine learning core consists of three blocks: a sensor data block, a computation block, and a decision tree (Figure 4). The sensor data block at the machine learning core aggregates data streams from the IMU’s internal accelerometers and gyroscopes, as well as any external sensors connected to the IMU via the I2C interface. The calculation block can filter the sensor data using predefined filtering parameters and calculate window statistics including the mean, variance, peak-to-peak amplitude, minimum value, maximum value, and zero-crossing of the sensor data. Decision trees compare sensor data statistics to thresholds to classify input data.
Figure 4: The machine learning core of STMicroelectronics’ LSM6DSO IMU consists of three blocks: a sensor data block, which aggregates data streams from internal and external sensors; a computation block, which performs filtering and statistical calculations on sensor data; and a decision tree , which categorizes activities based on computed statistics. (Image credit: STMicroelectronics)
As with the LSM6DSO’s FSM, dedicated tools in the Unico development environment program the IMU’s machine learning core.
Finite state machines and machine learning cores can also be used in conjunction with the host processor to implement more complex position tracking algorithms. STMicroelectronics’ downloadable X-CUBE-MEMS1 software package is available for the company’s STM32Cube development system and includes the following sample software routines:
Activity Recognition C provides information about the type of activity the user is performing, including staying still, walking, walking, jogging, cycling, or driving. This algorithm is often used in mobile phones or some wearable devices.
Motion Duration Detection C When used in conjunction with pedometer data, Motion Duration Detection can be used to determine seconds of user activity. This algorithm is commonly used in wearable fitness or health tracking devices.
Vibration or exercise intensity detection C provides information about the user’s exercise intensity, which can be recognized from 0 (resting) to 10 (full speed running). This algorithm is often used on mobile phones or some wearable fitness devices.
Carrying Position Recognition C provides information on how the user carries the device, distinguishing between the following positions: resting on a desktop, in hand, close to the head, in a shirt pocket, in a trouser pocket, in a jacket pocket, and worn on a swinging arm. The algorithm is often used on cell phones or other portable devices to detect activity-related context.
Summarize
To maintain positioning and use IMU data to detect motion and gestures, it is necessary to keep the host processor running continuously, which can be difficult for battery-powered embedded designs because the power consumption of the host processor is quite high. However, a new generation of low-power IMUs with on-board processing power sufficient to perform machine learning solves this problem by allowing the host processor to sleep in a low-current mode and wake up when necessary.
The Links: KCS057QV1BL-G21 CM100DY-24NF TIMMALCD