improve memory streams parsing

[1ndahous3]

1. Add support of MemoryList stream type.
2. Add limited support of dumps without MemoryInfoList.

This implements parsing of some incomplete/synthetic dumps.

DRAKVUF minidump to check (added extra .zip extension to satisfy dumb github):

• 9f950b5ede3745ce9a40d17a6e83c34d52cbf3995bbce767448104db1d08e263.zip.001.zip
• 9f950b5ede3745ce9a40d17a6e83c34d52cbf3995bbce767448104db1d08e263.zip.003.zip
• 9f950b5ede3745ce9a40d17a6e83c34d52cbf3995bbce767448104db1d08e263.zip.002.zip

[0vercl0k]

Woot, this looks awesome! Give me a week or so to take a look and try it out 🙏 Cheers

…

On Sun, Feb 25, 2024 at 5:18 AM Roman ***@***.***> wrote: 1. Add support of MemoryList stream type. 2. Add limited support of dumps without MemoryInfoList. This implements parsing of some incomplete/synthetic dumps. DRAKVUF minidump <https://github.com/tklengyel/drakvuf/blob/750a51f844b5b914051707e71f4444dbe95a7417/src/plugins/procdump/minidump.h#L673-L683> to check (added extra .zip extension to satisfy dumb github): - 9f950b5ede3745ce9a40d17a6e83c34d52cbf3995bbce767448104db1d08e263.zip.001.zip <https://github.com/0vercl0k/udmp-parser/files/14396437/9f950b5ede3745ce9a40d17a6e83c34d52cbf3995bbce767448104db1d08e263.zip.001.zip> - 9f950b5ede3745ce9a40d17a6e83c34d52cbf3995bbce767448104db1d08e263.zip.003.zip <https://github.com/0vercl0k/udmp-parser/files/14396438/9f950b5ede3745ce9a40d17a6e83c34d52cbf3995bbce767448104db1d08e263.zip.003.zip> - 9f950b5ede3745ce9a40d17a6e83c34d52cbf3995bbce767448104db1d08e263.zip.002.zip <https://github.com/0vercl0k/udmp-parser/files/14396440/9f950b5ede3745ce9a40d17a6e83c34d52cbf3995bbce767448104db1d08e263.zip.002.zip> ------------------------------ You can view, comment on, or merge this pull request online at: #16 Commit Summary - b655a5d <b655a5d> implement MemoryList stream parsing, parse dumps without MemoryInfoList stream File Changes (1 file <https://github.com/0vercl0k/udmp-parser/pull/16/files>) - *M* src/lib/udmp-parser.h <https://github.com/0vercl0k/udmp-parser/pull/16/files#diff-98aee139cd8fee86595b1f7876b920cd8bb2b1b120d7837dfef8401f2ff86c88> (132) Patch Links: - https://github.com/0vercl0k/udmp-parser/pull/16.patch - https://github.com/0vercl0k/udmp-parser/pull/16.diff — Reply to this email directly, view it on GitHub <#16>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALIORN3ZQIILB5UVQFIAL3YVM2ZZAVCNFSM6AAAAABDY3XWIKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGE2TENZWGIZDAMA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[0vercl0k]

All right sorry for the lag on this - looking now.

[0vercl0k]

Ok so does this add support for the MemoryList stream but those dumps don't have a MemoryInfo? I am a bit confused about this code added?:

      //
      // If we don't find an existing entry, we can save the memory region here,
      // but the meta information will be invalid
      //

      auto It = Mem_.find(StartOfMemoryRange);
      if (It == Mem_.end()) {
        DbgPrintf("The memory region starting at %" PRIx64
                  " does not exist in the map, the meta information will be "
                  "invalid.\n",
                  StartOfMemoryRange);

        dmp::MemoryInfo_t MemInfo;
        MemInfo.BaseAddress = StartOfMemoryRange;
        MemInfo.RegionSize = DataSize;

        auto Emplace = Mem_.try_emplace(StartOfMemoryRange, MemInfo);
        if (!Emplace.second) {
          DbgPrintf("The region %" PRIx64 " is already in the memory map.\n",
                    CurrentData);
          return false;
        }

        It = Emplace.first;
      }

The issue with this, is that the end user has no way programmatically to know what is 'valid' or 'invalid' which isn't great. Also, it seems like 'regular' (created by Windows tools) wouldn't create a dump with memory list entries and no matching info list?

Cheers

[1ndahous3]

Ok so does this add support for the MemoryList stream but those dumps don't have a MemoryInfo?

Yes exactly. Technically, the streams containing the memory layer (raw data) are "MemoryList" and "Memory64List", so minidumps without the "MemoryInfoList" stream still contain all the necessary information to access the memory blocks.

I don't really know the correct set of streams in the "correct" minidumps, but I see projects that use the minimum set of streams to store data, and this patch adds the ability to work with such dumps.

I tried parsing minidumps of the DRAKVUF project (which I mentioned in the first post, see linked code + minidump) and realized that this parser can parse data, but with this small patch.

The issue with this, is that the end user has no way programmatically to know what is 'valid' or 'invalid' which isn't great

I'm sure that DRAKVUF is not the only project with a non-standard minidump, so I think it's better to support it (and print warnings) than not to support it. Also truly bad/broken minidumps will be filtered by other checks.