Analyzing events with Ebrow

Index

Chronological tab
Thumbnails tab
Screenshot tab
Power plot tab
Color mapped 2D plot tab
Perspective 3D plot tab
Events details
Fake events gallery
Conclusions

Chronological tab

The Events tab is also divided into several tabs; the first of them contains the Chronological table of the events. The table can be scrolled both horizontally and vertically. It derives from the automatic_data database table, after adding the classifications. Since the columns are the same, their descriptions can be found here.
Events->Chronological tab
Events->Chronological tab

The left panel contains controls and informations related to the showed tab only.
For Chronological tab it is possible to select only the events whose classification satisfies the filters set in the left panel. For this reason, the numbering of the events under the ID and DAILY_NR columns are not contiguous: the figure shows that the overdense filter has been set, for which only events lasting longer than 1 second are listed. This is in fact the default setting for the minimum duration of an overdense event, but it can be changed from the Preferences tab. If no filters are checked, no rows will appear in the table. The filter setting is stored in the ebrow.ini file and is therefore persistent between successive analysis sessions.

Another thing to note in the above image is that only the leftmost Chronological and rightmost Shot Details tabs are active. This happens because, to contain the size of the database file, the parameter Images Lasting has been set in Echoes to automatically delete images and dump files older than N days, so those of event #1 have already been deleted and consequently there are no more screenshots or even graphics to show relating to that event.
The same thing also happens when selecting the latest events, those of the current day, because before performing the midnight swap the images and dump files have not yet been loaded into the database and are still in the file system.

The tab Screenshot appears only if in the DB, for the selected event, there is the screenshot available, otherwise it remain grayed.
The same thing can be said about the three plot tabs at its right (Power, 2D, 3D), that will be enabled only when there the selected event is provided with a dump file, since the plots are based on them.

By pressing the Export button, the entire contents of the history table are saved in the working directory, under ebrow/exports/events/, in a CSV file. Before saving, the program asks to enter a comment, which will then be saved in a .txt file named comments_chronological_<from date>_to_<to date>.txt.

The first time the Export button is pressed, Ebrow checks if there are already files in the export/ directory and asks if they should be deleted before adding the new file or not. If not, the files already present will be included in the report, if a report will be requested.


Thumbnails tab

This tab displays the previews of the events that occurred on the same day as the one selected in the Chronological tab, still according to the filters set. The thumbnail management needs better optimization because the creation and loading of hundreds of thumbnails takes several tens of seconds, but it is still very useful for selecting the most interesting events. If no screenshots are available but dumps are, the Thumbnails tab will display their daily ID labels, without images.
Events->Thumbnails ta
Events->Thumbnails tab

By pressing the Export button, the screenshots related to all the thumbnails showed will be saved in the working directory, under ebrow/exports/events/.
If the DAT box is checked, along the screenshots will be saved also the relative dump files if present. Since release 0.1.61, both dump formats, the old ASCII .DAT and the newer binary .DATB are supported.

Screenshot tab

Opening the Screenshot tab, the full size screenshot of the selected event is displayed.
On the left panel, a Plot Settings box has appeared, where the two Zoom sliders allow to magnify the image differently on the two vertical and horizontal axes.
The Linked Sliders checkbox allows, when checked, to move both sliders by acting on only one of them.
The second checkbox, Show Grid, is not active in this tab.
Finally, the Reset pushbutton sets all the above controls to their default values.
Events->Screenshot tab
Events->Screenshot tab

By pressing the Export button, the current screenshot will be saved in the working directory, under ebrow/exports/events/.
Before saving, the program asks to enter a comment, which will then be saved in a .txt file named as the screenshot, prefixed with comment_. That comment will appear as the caption of the image when it is included in the HTML report.
If the DAT box is checked, along the screenshot will be saved also the relative dump file if present.

Power plot tab

Opening the Power plot tab, the program shows the event's power profile. The power profile shows the same information that we can see in the Power History graph, at waterfall window's right. However, keep in mind that - in the power history graph - the time flows upwards while the power profile is a classic XY graph with time advancing to the right on the abscissas, so the first is rotated by 90° and specular with respect to the second.
By placing the mouse on a plot, a balloon appears, showing the XY position and the dBfs value in that point. This balloon can be disabled in the Preferences tab.

On the left panel, there is the Plot Settings box where the two Zoom sliders allow to magnify the plot differently on the two vertical and horizontal axes.
The Linked Sliders checkbox allows, when checked, to move both sliders by acting on only one of them.
The second checkbox, Show Grid, when checked, draws a grid above the profile, at each notch.
Finally, the Reset pushbutton sets all the above controls to their default values.

Note that some controls have immediate effect, while others, like Show Grid, require to press the Refresh pushbutton to make effect.
Events->Power plot tab
Events->Power plot tab

By pressing the Export button, the power profile graph will be saved as PNG file in the working directory, under ebrow/exports/events/.

If the DAT box is checked, along the PNG will be saved also the relative dump file.
Before saving, the program asks to enter a comment, which will then be saved in a .txt file named as the screenshot, prefixed with comment_. That comment will appear as the caption of the image when it is included in the HTML report.

Color mapped 2D tab

This type of graph reconstructs, starting from the data present in the dump file, the information shown in the screenshot but in a more versatile way. Unlike the screenshot, the graph can be rendered and edited on the fly, changing the view thresholds, zoom and color palette for a more effective graphic result. Besides this, the dump file can cover a longer time interval, regardless of the waterfall window's size. This is what the control Data Dump Coverage does.
By placing the mouse on the graph, a balloon appears, showing the XY position and the dBfs value in that point. This balloon can be disabled in the Preferences tab.

On the left panel we have a Plot Settings box, showing the Total Dynamic Range of the graph first. In the image below, the lowest power point is -135 dBfs while the highest is -45 dBfs.

Below these infos, there are two sliders to restrict the Visible Dynamic Range: they allow to set the power values from which the colored scale begins and ends, the minimum and maximum values displayed. The other points having power values not included in this interval, will be represented with the first value of the scale if lower than it, and with the last one if higher. In the figure, all values between -135 and -85 will be displayed as -85, as dark blue points.

Just below that, we have the two Zoom sliders that allow to magnify the graph differently on the two vertical and horizontal axes.
The Linked Sliders checkbox allows, when checked, to move both sliders by acting on only one of them.
The second checkbox, Show Grid, when checked, draws a grid above the profile, at each notch.

The Colormap control selects the color palette to be used for drawing. Among those available, there is the Echoes palette which is the one used for the waterfall window, then the Colorgramme which is the palette used in RMOB graphs. The remaining palettes are a selection of those made available by the Python's Matplotlib library used to draw graphs.
Finally, the Reset pushbutton sets all the above controls to their default values.

Note that some controls have immediate effect, while others, like Colormap, require to press the Refresh pushbutton to make effect.
Events->Color mapped 2D tab
Events->Color mapped 2D tab

By pressing the Export button, the 2D graph will be saved as PNG file in the working directory, under ebrow/exports/events/.

If the DAT box is checked, along the PNG will be saved also the relative dump file.
Before saving, the program asks to enter a comment, which will then be saved in a .txt file named as the graph, prefixed with comment_. That comment will appear as the caption of the image when it is included in the HTML report.

Perspective 3D tab

The 3D graph is always impressive, at least in presentations. Unlike 2D graphs, the 3D graph can be rotated both horizontally (Azimuth) and vertically (Elevation) on the fly. The Plot Settings box is identical to that of 2D plots, but an additional 3D View Settings box is added here where the sliders for Azimuth and Elevation moving are placed. The same effect can be obtained by dragging the figure with the mouse.

Note that some controls have immediate effect, while others, like Colormap, require to press the Refresh pushbutton to make effect.
Events->Perspective 3D tab
Events->Perspective 3D tab

By pressing the Export button, the 3D graph will be saved as PNG file in the working directory, under ebrow/exports/events/.

If the DAT box is checked, along the PNG will be saved also the relative dump file.
Before saving, the program asks to enter a comment, which will then be saved in a .txt file named as the graph, prefixed with comment_. That comment will appear as the caption of the image when it is included in the HTML report.

Event details

Events-Details
Events->Details

On this page, the detailed information of the selected event is shown rotated by 90° with respect to how it is read in the history, and is a little more comfortable to read. This table can also be exported as CSV.

The Fix Classification box on the right is interesting; here the user can correct a classification he deems wrong. If the database will be saved at the end of the operations, the new classification will remain stored in the database and reloaded when the database is reopened.

Fake events gallery

In this paragraph I describe the types of false positives that can appear among the captured events and the criteria with which the related classification filters operate. The filters process the data recorded, for each event, in the table automatic_data.
ESD fakes
The disturbance showed in the image appears at the evening, every time I switch on/off my desk lamp. Other types of ESD disturbance are caused, for example, by the electric motors of blenders or hair dryers and consist of bundles of horizontal lines similar to those in the figure which are irregularly repeated.
False positive due to artificial electrostatic discharge
False positive due to artificial electrostatic discharge

More significant ESD disturbances are detected during thunderstorms, caused by lightnings, like in the sample below:
False positive due to lightnings
False positive due to lightnings

The ESD filter uses the diff_start and diff_end columns to detect false positives. When the average of these two values calculated during the peak, approaches the upper threshold with a percentage tolerance equal to ESD filter threshold, the event is considered a fake.
Saturation fakes
The presence of transmissions near the frequency tuned by the receiver - of great power or less power but located close to the receiver - can cause its saturation. This also happens because to receive meteor echoes it is generally necessary to set the highest possible RF gain.
This effect is caused by the receiver's AGC, which reduces sensitivity automatically in proportion to the strength of the received signal. Although Echoes disables AGC when you set a RF Gain value other than Auto, RTL-SDR receivers appear to ignore this setting and are therefore prone to these false positives.
False positive due to receiver saturation
False positive due to receiver saturation

This filter works by storing the average S recorded on any event (avgS) that becomes a reference for the next one. If - in the next event - the avgS falls below that reference, exceeding the given tolerance percentage, the event is marked as fake.
RFI fakes
These disturbances have the same origin as saturations, with the difference that this time the interfering signal is received with less power and does not saturate the receiver, but still manages to produce false positives.
An analogue radio station that broadcasts voice and music can produce such interference continuously, so much so that one might almost not notice its presence if one does not first check the spectrum around the working frequency. However, its presence increases the value of N, consequently lowering the sensitivity of our station for the weakest echoes.
In these cases, the only thing that can be done is to act on the quality of the receiving antenna and its shielded cable and place hardware filters on the line to block the disturbing transmission. Very little can be done with software only.

Radio interferences can also appear suddendly. For example, starting from the end of January 2024, the GABB.IT station began to record an anomalous amount of events at around 20 UTC every Wednesday:
GABB.IT RMOB data, February 2024
GABB.IT RMOB data, February 2024

By analyzing the spectra corresponding to these events, I found a large quantity of spectra similar to the one in the following image. Don't be fooled by the vertical line, that signal is outside the detection range and is of no importance in this case. What happened is that radio interference quickly raised the value of N and the S-N difference, represented in the vertical diagram, with the orange trace, causing the capture to start. During the capture, the automatic thresholds remain fixed until the capture ends, but the difference now bounces between the thresholds causing a single long-lasting event.
False positive due to ongoing voice transmission close to the tuned frequency
False positives due to ongoing voice transmission close to the tuned frequency

In order to correctly detect these events it is necessary to work with a version of Echoes equal to or higher than 0.55 because the RFI filter is based on the std_dev column, which until v.0.54 contained the standard deviation of the S-N difference while following tests I saw that the standard deviation of N is more useful for this purpose. In fact, the relative filter discards as fakes all events that have a std_dev higher than the RFI filter threshold. This threshold by default is 1.0 but you can also set a lower value such as 0.9. Going below this value risks discarding good events as fakes, but a lot also depends on your setup and the frequency of your signal source.

In the case of the event shown below, a transmission occurred at a lower frequency than the tuned one and its interference caused the upper threshold to be exceeded and the event was captured
False positive due to ongoing packet transmissions close to the tuned frequency
False positives due to ongoing packet transmissions close to the tuned frequency

This type of interference could be discriminated more easily with the ESD filter than with the RFI, but depending on the thresholds set, it may not even be recognized as such.
Carrier fakes
The following image illustrates what I mean by carrier fake. This is an artificial non-modulated signal or one of its harmonics that suddenly appears within the detection area, triggering an event. The carrier in the image in fact is a well known one, being the GRAVES radar carrier, but let's forget this now and assume it's a temporary signal we need to get rid. If the event persists for a long time, Echoes may remain in the capturing state indefinitely. In practice, just to overcome this problem, Echoes forcibly closes an event that exceeds the parameter Cancel Events Lasting More Than (default 50 seconds).
Given that the presence of the disturbance also alters the N value, when automatic or differential thresholds are set, Echoes no longer detects new events until that disturbance disappears. Conversely, with absolute thresholds, a new event would be detected immediately, ending up to collect a large quantity of false positives.
False positive due to carrier-like signal or its harmonics
False positive due to carrier-like signal or its harmonics

Ebrow provides two filters to detect these false positives which operate on two different criteria.

The first one works on the ratio between echo area and interval area because the carriers occupy only a very small part of the detection area, just a vertical line as can be seen from image above. Must be said that many echoes can go very close to this aspect, so if the relationship between the two areas falls below the Carrier filter 1 threshold, the event is marked as fake.

The second filter discriminates the carrier fakes looking at the difference between instantaneous difference S-N and its average value. The event is marked as fake when When this difference, expressed as percentage, falls below the Carrier filter 2 threshold.
Conclusions
Even if the filters manage to remove a large part of the false positives, the fact remains that when these are detected, any true echoes are lost and this is especially true for fake carriers and fake RFIs which can block detection for a long time if the disturbance persists. For this reason you need to pay attention to the disturbances detected by your station, their quantity, duration and repetition over time because in the end they will tend to alter your counts. Hardware solutions to problems resulting from disturbances are always preferable.
  • Back
  • Home
  • Next