Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I normalize temporal data to their initial values?

I have data that are acquired every 3 seconds. Initially, they always begin within a narrow baseline range (i.e. 100±10) but after ~30 seconds they begin to increase in value.

Here's an example.

enter image description here

The issue is that for every experiment, the initial baseline value may start at a different point in the y-axis (i.e. 100, 250, 35) due to variations in equipment calibration.

Although the relative signal enhancement at ~30 seconds behaves the same across different experiments, there may be an offset along the y-axis.

My intention is to measure the AUC of these curves. Because of the offset between experiments, they are not comparable, although they could potentially be identical in shape and enhancement ratio.

Therefore I need to normalize the data so that regardless of offset they all have comparable baseline initial values. This could be set to 0.

Can you give me any suggestions on how to accomplish the normalization on Matlab?

Ideally the output data should be of relative signal enhancement (in percent relative to baseline).

For example, the baseline values above would hover around 0±10 (instead of the raw original value of ~139) and with enhancement they would build up to ~65% (instead of the original raw value of ~230).

Sample data:

 index       SQMean   
_____    ____________

'0'      '139.428574'
'1'      '133.298706'
'2'      '135.961044'
'3'      '143.688309'
'4'      '133.298706'
'5'      '133.181824'
'6'      '134.896103'
'7'      '146.415588'
'8'      '142.324677'
'9'      '128.168839'
'10'     '146.116882'
'11'     '146.766235'
'12'     '134.675323'
'13'     '138.610382'
'14'     '140.558441'
'15'     '128.662338'
'16'     '138.480515'
'17'     '153.610382'
'18'     '156.207794'
'19'     '183.428574'
'20'     '220.324677'
'21'     '224.324677'
'22'     '230.415588'
'23'     '226.766235'
'24'     '223.935059'
'25'     '229.922073'
'26'     '234.389618'
'27'     '235.493500'
'28'     '225.727280'
'29'     '241.623383'
'30'     '225.805191'
'31'     '240.896103'
'32'     '224.090912'
'33'     '230.467529'
'34'     '248.285721'
'35'     '233.779221'
'36'     '225.532471'
'37'     '247.337662'
'38'     '233.000000'
'39'     '241.740265'
'40'     '235.688309'
'41'     '238.662338'
'42'     '236.636368'
'43'     '236.025970'
'44'     '234.818176'
'45'     '240.974030'
'46'     '251.350647'
'47'     '241.857147'
'48'     '242.623383'
'49'     '245.714279'
'50'     '250.701294'
'51'     '229.415588'
'52'     '236.909088'
'53'     '243.779221'
'54'     '244.532471'
'55'     '241.493500'
'56'     '245.480515'
'57'     '244.324677'
'58'     '244.025970'
'59'     '231.987015'
'60'     '238.740265'
'61'     '239.532471'
'62'     '232.363632'
'63'     '242.454544'
'64'     '243.831161'
'65'     '229.688309'
'66'     '239.493500'
'67'     '247.324677'
'68'     '245.324677'
'69'     '244.662338'
'70'     '238.610382'
'71'     '243.324677'
'72'     '234.584412'
'73'     '235.181824'
'74'     '228.974030'
'75'     '228.246750'
'76'     '230.519485'
'77'     '231.441559'
'78'     '236.324677'
'79'     '229.935059'
'80'     '238.701294'
'81'     '236.441559'
'82'     '244.350647'
'83'     '233.714279'
'84'     '243.753250'
like image 453
pepe Avatar asked Nov 20 '25 20:11

pepe


2 Answers

Close to what was mentioned by Shai:

blwindow = 1:nrSamp;
DataNorm = 100*(Data/mean(Data(blwindow))-1)

Set the window to the right size, however you want to determine it, it depends on your data. Output DataNorm is in %.

like image 154
Wooly Jumper Avatar answered Nov 23 '25 12:11

Wooly Jumper


Usually this kind of problems requires some more specific knowledge about the data you are measuring (range, noise level, if you know when the actual data starts etc.) and the results you are trying to achieve. However, based on your question only and by looking at your example graph, I'd do something like this (assuming your data is in two arrays, time and data):

initialTimeMax = 25; % take first 25 s
baseSample = data(time <= initialTimeMax); % take part of the data corresponding to the first 25 s
baseSampleAverage = mean(baseSample); % take average to deal with noise
data = data - baseSampleAverage;

If you don't know when your data starts, you can apply a smoothing filter, then take a derivative, find the x-position of its maximum, and set initialTimeMax to this x-position.

like image 37
texnic Avatar answered Nov 23 '25 11:11

texnic



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!