There is a way to Mouse Click on Entry without clicking on info button


#21

:slight_smile: je sais, mais bon …


#22

@Viir is it possible with custom region replacement to replace only source region but not destination region while DragDrop?

I’m testing this:

		static public MotionParam MouseDragAndDropOnCustomRegion(
			this IUIElement elementToDrag,
			IUIElement destination,
			RectInt region,
			MouseButtonIdEnum mouseButton) =>
			new MotionParam
			{
				MouseListWaypoint = new[] { elementToDrag, destination }.Select(uIElement => new MotionParamMouseRegion() { UIElement = uIElement, RegionReplacement = region })?.ToArray(),
				MouseButton = new[] { mouseButton },
			};

but obviously with this call:

var region = RectInt.FromMinPointAndMaxPoint(new Vektor2DInt(20, 20), new Vektor2DInt(100, 40));
return testwindow?.MouseDragAndDropOnCustomRegion(testdesto, region, MouseButtonIdEnum.Left);

it’s correctly changing region on testwindow but as well on testdesto, which is not wanted. I guess
"RegionReplacement " works for both, is there any other option?

Thanks!


#23

Yes: The source region is the first element you pass into the parameter MouseListWaypoint. So you want to only replace the region for this first element in this list, not the other ones.

I have adapted this to replace only the source region:

static public MotionParam MouseDragAndDropFromCustomRegion(
	this IUIElement originElement,
	RectInt originRegion,
	IUIElement destination,
	MouseButtonIdEnum mouseButton) =>
	new MotionParam
	{
		MouseListWaypoint = new[] { new MotionParamMouseRegion() { UIElement = originElement, RegionReplacement = originRegion }, new MotionParamMouseRegion() { UIElement = destination }},
		MouseButton = new[] { mouseButton },
	};

CCP broke their EVE Launcher, so not tested live.


#24

works perfectly, thanks!


#25

@viir one more question about regions

How they are defined? eg. are they absolute to eve screen or relative to something?

because when we take example of the InfoPanelCurrentSystem (blue on image), it has got min max 32,71,344,99 (when folded) and it kind of correlate with pixels counted from 0,0 without Neocom panel - so I guess they are relative to left top corner without Neocom?

if I try to substitute it with

var region = RectInt.FromMinPointAndMaxPoint(Header.Region.MinPoint(), new Vektor2DInt(100, 99));
yield return Header?.MouseDragAndDropFromCustomRegion(region, Desto, MouseButtonIdEnum.Left);

I’m kind of expecting to be in same region but shorder by 244 pixels.
But in reality I will end up somewhere else, more like around Route marker.

So then I was thinking to try something simple, like

var region = RectInt.FromMinPointAndMaxPoint(new Vektor2DInt(0,0), new Vektor2DInt(20,20));

But that sent mousecursor roughly to red area on sample picture, which is unexpected as my thoughts was cursor will be at left top corner in square 0,0 20,20

Am I missing something from rect definition?
Thanks

tempsnip


#26

I am looking at the API now and I find multiple types of regions:

MemoryStruct.UIElement.Region

To find the definition, I am looking for assignments to the UIElement.Region property in the memory reading source code.
Here I find the following candidate which looks most promising to me for now:

Following the code in FläceAusGbsAstInfoMitVonParentErbe, I guess it is an aggregation of offsets of all parent nodes. But your post looks you are more interested in RegionReplacement so I will prioritize that now.

Sanderling.Motor.MotionParamMouseRegion.RegionReplacement

To find the definition of RegionReplacement I look in the source code for applying the effects to the EVE Online client, and find this:

Reading this code, it seems like: RegionReplacement is defined relative to the coordinate system used by UIElement.Region, which you seem to have a definition for already:

The difference I see in this code is that the RegionReplacement has another offset. It is centered at the original Region of the UIElement. In other words, when you specify a RegionReplacement centered at (0,0), it should come out having the same center at the UIElement.Region.
Is this model consistent with your observations?

Maybe it would be better if the Region would remain the same when the Region is used as RegionReplacement :thinking:


#27

thanks for comprehensive answer @Viir :wink:

my observation with replacements are different, I did expected two variations would happen
a] replacement region works from left-top corner of EVE screen
b] replacement region work from lett-top of given UI element.

Unfortunately on given example I’m for some very unknown reason able to hit correct area-ish only when my offset looks like this

var region = RectInt.FromMinPointAndMaxPoint(new Vektor2DInt(-110,-20), new Vektor2DInt(-70,0));
yield return Header?.MouseDragAndDropFromCustomRegion(region, desto, MouseButtonIdEnum.Left);


static public MotionParam MouseDragAndDropFromCustomRegion(
	this IUIElement originElement,
	RectInt originRegion,
	IUIElement destination,
	MouseButtonIdEnum mouseButton) =>
	new MotionParam
	{
		MouseListWaypoint = new[] { new MotionParamMouseRegion() { UIElement = originElement, RegionReplacement = originRegion }, new MotionParamMouseRegion() { UIElement = destination }},
		MouseButton = new[] { mouseButton },
	};

That’s why I’m more curious about how this is even possible :slight_smile: eg. regionreplacement seems to be working outside given UIElement region and on top of it with strange offset.
Would be great if you or somebody can verify it on different code or region


#28

@pikacuq, reading this makes me wonder if the current API for specifying where to place the mouse is too complex.
Could we avoid this problem by enabling the bot to specify concrete points relative to the upper left corner EVE Online window?
What I have in mind is having to bot give a two dimensional vector for each starting point and end point of the drag&drop operation.


#29

I wouldnt say it’s complex - because replacement is quite straightforward, but definitely it’s not clear with which grid bot is working and therefore coordinates is big guess game.
I think there are two easy ways.

1] when element has got own region, replacement may work INSIDE given element, so you can shorten it, but not overlap with other elements (which is basically what many may need)

2] work with relative grid to left upper corner of eve window. Which would be easier to measure in any graphical program so everybody would know what region he is working in based on exact coordinates.
This would be handy to specify exact areas of interest no matter which element is currently there. Of course there is a downside with counting them correctly when window size is changed.

I think, 2nd is more powerful and gives people more flexibility, but they have to be aware of window size, when 1st gives them kind of flexibility but in borders of given regions.and no need to worry about window size.
Personally shortening given element by several pixels would be enough functionality.


#30

At the moment, I prefer this option. One reason is to further reduce the ‘guess game’ problem mentioned earlier. By moving all the constraints and mappings into the bot code, we support learning about them using the time machine. (The time machine lets you look back in time at how the bot made decisions and computed the effects to apply to the EVE Online client).

I am adding option 2 to the backlog for now.

I don’t understand what are you thinking about:

  • What is counted? Are the coordinates counted?
  • Are you thinking of the window size change happening between the memory measurement and the application of the mouse drag effect? Or does the change happen at some other time?

Most importantly, I do not understand how the window size change would be handled differently in option 1.
If the window size change happened between measurement and mouse input, I would expect option 1 to exhibit the same problem.
If the window size change happened before the memory measurement, the changed window size will be already in the memory measurement, so the bot would use those if it needs to consider the window size.


#31

when somebody will be using coordinates from variant 2] … let’s say something on right side will have coordinates defined in the code 1200 20 1250 60 and then you’ll resize your eve window to smaller width, it might cause issues because coded values will be outside actual window.

as first mentioned method would let people to override coordinates completely based on left top grid, they might run into troubles when they define some coordinates based on actual window width and then width will change.

in 1 variant all regions are recalculated when windowsize is changing you cannot really be in that situation as you are already working inside defined coordinates eg. Element regions.

That is why I mentioned it would be safer for end-coders to code scripts as they dont have to worry about eve window size.

Dunno if Iam explaining myself correctly :wink:


#32

I did not find a concrete example which fits this description.

If anybody sees a problem with changing the API to take the mouse coordinates from the bot as relative to the upper left corner EVE Online window, I recommend showing a concrete scenario where this approach would cause a problem.


#33

let’s say you are switching tabs on the overview which is on the right side of fulscreen eve.
Then you create another profile which will have 1200x1024 windowsize instead of fullscreen 1900x1200 or something.

your region for tabs will be out of new window size hardcoded in the region replacement, wont be?

If not, then it’s fine.


#34

Yes, the best idea I found so far for the API:

  • Have the bot specify an exact point for the mouse location using a two dimensional vector. Yes, this allows a bot to give coordinates outside the window to the framework. When the bot specifies a point outside the window as a mouse location, the framework skips this point.
  • If the bot author wants to avoid giving a point outside the window to the framework, they would have to add a branch to the bot code. The bot can ask this question:

Is this location inside the game window?

Based on the answer, the bot knows if the framework will use or ignore this mouse location.

Ideas for improving the API are welcome.


#35

That makes sense, if there will be some sort of feedback it should be safe to do so.

Thanks!


#36

Regarding the feedback, I see these two ways, for different times:

  • Early, in the bot step: The bot can ask Is this location inside the game window? and continue based on the frameworks answer.
  • Later, after that bot step, or after the complete botting session: A user can use the time machine with the session recording to inspect an individual bot step. There, the user can inspect the effects which the bot gave to the framework in this step. The user can also look at the memory measurement to look up the window size in this bot step.

As an additional possible feature, the session recording could also be automatically analyzed to answer this question:

When did the bot ask the framework to move the mouse to locations outside the game window?

The answer could be provided in form of a list of bot steps in which this was the case. This automated way to “look for possible problems in a bot implementation” is just an expansion on the manual process of using the time machine to inspect individual bot steps.