Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IA2 dump tree implementation #75

Merged
merged 4 commits into from
Feb 29, 2024
Merged

IA2 dump tree implementation #75

merged 4 commits into from
Feb 29, 2024

Conversation

spectranaut
Copy link
Collaborator

@spectranaut spectranaut commented Feb 20, 2024

In this PR:

  • Temporarily: when you pass a PID to the "dump_tree_ia2" function, it will only fine the relevant HWND for "Google Chrome". Eventually I'll remove the PID option and find the HWND by string, alone.
  • I added the IAccessible2 header and interface constant files ia2_api_all.h and ia2_api_all_i.cc that was created by running concat.sh in LinuxA11y/IAccessible2 then MIDL
  • You can dump the tree with: build/bin/Release/dump_tree_ia2.exe

Follow up tasks:

  • get any window, not just google chrome. probably pass in a string to search for?
  • experiment with where to put CoInitialize(nullptr);, because you have to call CoUninitialize() when there are no longer any interface pointers on the stack (otherwise you get a seg fault). you can call multiple times -- as long as its the same amount of times. possibly we can to call every time we create or destroy a "IA2Node" ?
  • we should probably make wrapper objects for the interfaces, IAccessible2, IAccessibleText, etc. We might want to rename IA2Node to IANode.

@spectranaut spectranaut changed the title DRAFT: Try to get the IAccessible2 interface DRAFT: Trying to access the IAccessible2 interface Feb 20, 2024
@spectranaut spectranaut force-pushed the iaccessible2 branch 2 times, most recently from 2c9b723 to f5adbb0 Compare February 23, 2024 00:59
@spectranaut spectranaut changed the title DRAFT: Trying to access the IAccessible2 interface IA2 dump tree implementation Feb 27, 2024
@spectranaut spectranaut requested a review from alice February 27, 2024 18:47
Copy link
Collaborator

@alice alice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall - loads of comments/questions but nothing bad :)

examples/ia2/dump_tree_ia2.cc Outdated Show resolved Hide resolved
include/axaccess/ia2/ia2_api_all.h Show resolved Hide resolved
include/axaccess/ia2/ia2_node.h Show resolved Hide resolved
include/axaccess/ia2/ia2_node.h Outdated Show resolved Hide resolved
CMakeLists.txt Outdated Show resolved Hide resolved
lib/ia2/win_utils.cc Outdated Show resolved Hide resolved
// We will need these for windows related things, see the formatter_win.cc in
// chromium
// #include <oleacc.h>
// #include <wrl/client.h>

// TODO: we will probably need to get these from the IAccessible2 git repo, like
// in Chromium
// #include "third_party/iaccessible2/ia2_api_all.h"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again maybe not in this PR, but it might be nice to have a similar third_party convention.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true, it seems weird to just have these generated files here. made an issue: #96

lib/ia2/win_utils.cc Outdated Show resolved Hide resolved
HWND hCurWnd = nullptr;
// TODO: This is not the right API -- probably we should copy chrome code
// and find a hwnd by titla alone. Right now this only returns a Google Chrome
// HWND. But if I don't include PID, then I get other processess with google
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to filter on name here at all? Do we get multiple processes from the same ProcessID?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We get multiple windows for a single process id. And when I got all the windows (not scoped to a single process id) some of them also had "Google Chrome" in the in the name, I think it was the Google Chrome installer. But this will be handled in a follow up PR I think?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally fine, was just trying to understand what the underlying issue even is.

lib/ia2/win_utils.cc Show resolved Hide resolved
Copy link
Collaborator

@asurkov asurkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r=me with comments (including Alice's comments) addressed. I'm happy to see this working!

lib/ia2/ia2_node.cc Outdated Show resolved Hide resolved
lib/ia2/ia2_node.cc Outdated Show resolved Hide resolved

Microsoft::WRL::ComPtr<IServiceProvider> service_provider;
HRESULT hr = root_->QueryInterface(IID_PPV_ARGS(&service_provider));
hr = service_provider->QueryService(IID_IAccessible, IID_PPV_ARGS(&ia2));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps it should be moved to an utility method

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I agree this should be done, but I was thinking we'd do it in a follow up with #94 94 -- I added TODO note here!

Copy link
Collaborator

@alice alice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still looks good! Land when you're ready.

@spectranaut spectranaut merged commit 44e8003 into main Feb 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants