Skip to content

Commit

Permalink
Add documentation on log viewer and hard drive performance.
Browse files Browse the repository at this point in the history
  • Loading branch information
clovett committed Feb 16, 2017
1 parent e4e0a55 commit 29fdae2
Show file tree
Hide file tree
Showing 9 changed files with 84 additions and 1 deletion.
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ There are two ways you can generate training data from AirSim for deep learning.

![record screenshot](docs/images/record_data3.png)

There is a [Log Viewer](docs/log_viewer.md) provided that can visualize the streams of data that are published by the simulator (and your drone).

If you would like more data logging capabilities and other features, [file a feature request](https://github.com/Microsoft/AirSim/issues) or contribute changes. The data logging code is pretty simple and you can modify it to your heart's desire.

A more complex way to generate training data is by writing client code that uses our APIs. This allows you to be in full control of how, what, where and when you want to log data. See the next section for more details.
Expand Down
52 changes: 52 additions & 0 deletions docs/hard_drive.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Busy Hard Drive

There is a reason we recommend running your Unreal Environment on a Solid State Drive (SSD). Between debugging, logging, and Unreal asset loading
the hard drive can become your bottle neck. It is normal that your hard drive will be slammed while Unreal is loading
the environment, but if your hard drive performance looks like this while the Unreal game is running
then you will probably not get a good flying experience.

![Busy Hard Drive](images/busy_hard_drive.png)

In fact, if the hard drive is this busy, chances are the drone will not fly properly at all.
For some unknown reason this I/O bottle neck also interferes with the drone control loop and if that
loop doesn't run at a high rate (300-500 Hz) then the drone will not fly. Not surprizing, the control
loop inside the PX4 firmware that runs on a Pixhawk flight controller runs at 1000 Hz.

### Reducing I/O

If you can't whip off to Fry's Electronics and pick up an overpriced super fast SSD this weekend, then the
following steps can be taken to reduce the hard drive I/O:

1. First run the Unreal Environment using Cooked content outside of the UE Editor or any debugging environment,
and package the content to your fastest SSD drive. You can do that using this menu option:

![Package Unreal Project](images/package_unreal.png)

2. If you must use the UE editor (because you are actively modifying game assets), then at least
don't run that in a debugger. If you are using Visual Studio use start without debugging.
3. If you must debug the app, and you are using Visual Studio debugger, stop then Visual Studio from
logging Intellitrace information.
Go to Tools/Options/Debugging/Intellitrace, and turn off the main checkbox.
4. Turn off any [Unreal Analytics](https://docs.unrealengine.com/latest/INT/Gameplay/Analytics/index.html)
that your environment may have enabled, especially any file logging.

### I/O from Page Faults

If your system is running out of RAM it may start paging memory to disk. If your operating system has
enabled paging to disk, make sure it is paging to your fastest SSD. Or if you have enough RAM disable
paging all together. In fact, if you disable paging and the game stops working you will know for sure
you are running out of RAM.

Obviously, shutting down any other unnecessary apps should also free up memory so you don't run out.

### Ideal Runtime performance

This is what my slow hard drive looks like when flying from UE editor. You can see it's very busy,
but the drone still flies ok:

![Package Unreal Project](images/ue_hard_drive.png)

This is what my fast SSD looks like when the drone is flying in an Unreal Cooked app
(no UE editor, no debugger). Not surprizingly it is flying perfectly in this case:

![Package Unreal Project](images/cooked_ssd.png)
Binary file added docs/images/busy_hard_drive.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/cooked_ssd.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/log_viewer.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/package_unreal.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/ue_hard_drive.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
29 changes: 29 additions & 0 deletions docs/log_viewer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Log Viewer

The LogViewer is a Windows WPF app that presents the MavLink streams that it is getting from the
Unreal Simulator. You can use this to monitor what is happening on the drone while it is flying.
For example, the picture below shows a real time graph of the x, y an z gyro sensor information being generated by the
simulator.

### Usage

To use this LogViewer, connect the simulator `before` you run the simulation. Simply press the blue connector
button on the top right corner of the window, select the Socket `tab`, enter the port number 14388, and
your `localhost` network. Then press the record button (triange on the right hand side of the toolbar).
Now start the simulator, pick some mavlink items to graph, you should see something like this:

![Log Viewer](images/log_viewer.png)

The drone view here is the actual estimated position coming from the PX4, so that is a great way to check
whether the PX4 is in sync with the simulator. Sometimes you can see some drift here as the attitude
estimation catches up with reality, this is more visible after a bad crash.

### Installation

If you can't build the LogViewer.sln, there is also a
[click once installer](http://www.lovettsoftware.com/LovettSoftware/Downloads/Px4LogViewer/Px4LogViewer.application).

### Configuration

The magic port number 14388 can be configured in the simulator by editing the [settings.json file](settings.md).

2 changes: 1 addition & 1 deletion docs/prereq.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
### Computer
Intel Core i7 or equivalent PC with 32GB of RAM and a SSD and GPU with at least 4GB of RAM. Large environments in Unreal requires a lot of RAM and a beefy GPU card.
The typical computer hardware we use for development purposes usually have 6 to 12 cores and graphics card such as NVidia TitanX or NVidia GTX 1080 with 32GB to 64GB RAM
and fast SSD.
and fast SSD. See [why fast SSD](hard_drive.md).

### RC Transmitter and Receiver
This is the remote control that you usually use for real RC vehicles such as quadrotors. Our favorite is [FrSky Taranis X9D Plus](https://hobbyking.com/en_us/frsky-2-4ghz-accst-taranis-x9d-plus-and-x8r-combo-digital-telemetry-radio-system-mode-2.html) because it has built-in USB ports.
Expand Down

0 comments on commit 29fdae2

Please sign in to comment.