Top Page | Upper Page | Contents | About This Site | JAPANESE

Sensor Data Analysis

Sensor data of this page it that measured periodically, for example, by 1 second or 1 minute. Sensor data needs Time Series Analysis .

Big Data

Sensor data is big data. 1 minute data for 30 days needs 43220 lines (30 days * 24 hours *60 minutes). If the analysis software can deal with 65336 lines, we cannot analyze 2 months. If we want to analyze 1 second data, it is not easy.

"Big Data" is famous word recently. But Sensor data is also big data.

Repeat Type and Fluent Type

There are repeat type sensor data and fluent type in the plant data. Repeat type is made batch type process. Fluent type is continuty type process.
sensor data

Analysis of Sensor Data

Sensor data contains complicated and dynamic phenomena. Various time series analysis methods are needed to analyze sensor data.

You may think that the kind of Self Correlation Analysis is suitable for sensor data because raw sensor data has a column of time. But its mathematical model is not all-powerful.

In my experience, I often use Line Graph using Data Literacy and Measurement knowledge. It is an easy method. But it is reliable to analyze strange phenomena.

The Way to Time Analysis and Passing Analysis

When we use Analysis of The Time and The Hour and Passing Analysis for sensor data, processing of data is needed. We use the columns of time and a flag. The flag data tells us the timing of change,"ON and OFF".

We make difference of the time of the timing for the analysis of the hour.

We make new column of the difference between start time and timing time for passing analysis.

The Way to Multi-Variable Analysis and Data Mining

Multi-Variable Analysis and Data Mining methods are good at to analyze static phenomena.

If we process raw sensor data, these methods are powerful tools to analyze sensor data.

The process is sampling . It is the abstracting of special timing.

Decision Tree has the function to divide samples. But the function does not work well if the data contains complicated phenomena. Artificial sampling process as the previous step of the analysis helps the method.

Analysis of Sensor Data

Sensor data has complicated relationship and changing. So when we bring the data from database without any ideas, and put into multi-Variable analysis and data mining software, we cannot get what we want study.

Line Plot

I introduce the analysis of sensor data by the way, type 1, 1.5, 2 and 3. In these types basic tool is line plot.

It is not a high level tool. But it is useful the data we do not know the contains.

Process of Analysis

Repeat type is easier. So I write repeat type at first.

We need the idea of sampling and calculations before analysis.

NEXT Sampling from Database