In most scenarios, a lot of time is useless and taken by the human
inaction or slowness.
This optimizer tries to increase the scenario speed by removing this
wasted time. Stabilization is required before shots.
Note that animated tests such as label_slide cannot be optimized well.
The reason is that a stabilization is done at the beginning of the test,
resulting in an endless loop (slide always moving).
When changes are made in the source actions list (stabilize-shots...),
the modifications will be stored in the exu destination. This is not what we
want.
This patch fixes it by storing the original source scenario into the
destination exu.
It can happen that some tests take more time on weaker machines, leading
to wrong screenshots.
By using this option, the player will wait before each shot for the
application to be stabilized (no mode frames changes) and then take the shot.
It can happen if the file doesn't contain any actions or if no shot is
present. In this last case, an optimization in the player removes the
actions after the last shot, meaning it will remove all the actions if
no shot is present.
An example is window_states_2 test where the shots cannot be taken by
the recorder, as F2 is not caught.
To compare two exu files during the test suite, we need to unpack
the images in directories and make the comparison from there.
The problem is that the "orig" exu file was unpacked in the current
directory as well, leading to compare an image with itself.
The fix consists in unpacking the right exu file.
This can be useful if the application is not in an official repository
like elementary_test.
This feature allows compiling the source code and using it to run the
scenario on.
The command (e.g gcc...) must be stored in the unit as well. $SRC and
$DEST can be used to specify the source and destination files in the
command.
When the external injection is used (scenario injected by an external
tool via Eina Debug) and a screenshot is required, the data is sent
to the injector.
A bug appears in "box vert" test. If --disable-screenshots is set, and
no other optional such as -v, the application will receive only 4
clicked events instead of 8. The bug doesn't appear if -v is set or if
the application is shown on the screen.
It seems that the buffer engine doesn't want to consume the events
because no damage/shot request is "reported".
The solution for that bug is to enable the screenshots process, just
without saving the outputs.
@fix
Buffer evas cannot be used when the application is shown on the screen.
It means the _sync_shot_get function cannot work always. That is a
problem when the stabilization feature is used.
That's why we need to use snapshot no matter the case. The problem is
that snapshot seems to sit on the canvas and therefore doesn't forward
the events, such as mouse down... So we need to create a snapshot only
when needed.
Additionally, stabilization feature needed to be finished. The threshold
to guarantee the stabilization was not implemented.
One of the biggest issue in Exactness is related to the system
configuration differences. Among them, the fonts can for example
impact on the height of the widgets.
The solution to not be dependent on the fonts consist in using embedded
fonts and to force their usage when playing the applications.
The -f option has been added to the player and the recorder so the user
can provide the path to a fonts directory. This option must be set in
order to force the fonts replacement. Since tests shots can use different
fonts, the exu file stores the version of fonts that have been used.
This is why it is needed to have in the provided directory different
directories, each pointing to a different version of the fonts. For
example, some old tests can use fonts of 2017 (e.g directory 20170101)
while new tests will use new fonts (20180601). Check the
exactness-elm-data repository (fonts branch) for a better understanding.
During recording, the -f option will apply the indicated fonts on the
launched application and will record the mouse events accordingly. The
fonts datestamp is stored in the exu output.
During playing, the fonts will be loaded by reading the exu fonts path,
and then the application is launched. If no information is provided in
the exu but -f is used, the tool will load the most recent fonts (by
comparing the datestamp directories).
The problem that we try to solve is the time taken to compare the shots after
all the scenarios have been run.
Now, comparing the shots sequentially is done right after the test
finished to run. With -j 1, it won't change anything. With more CPUs, it
will compare while other tests are running, i.e when the CPU is not too
much busy.
The goal is to support applications where editable entries are used.
The problem is the text cursor that, even if we disable its animation
through the theme overlay, triggers the render post event, which breaks
all the previous method used to detect stability.
Now, every 100ms, we compare the current canvas image with the previous
saved shot.
The text cursor creates noise during the shots, as it is an animation.
In order to solve the issue, the theme is overlayed to hide the cursor.
Most of the entry edc file must be overlayed (and not only the cursor
group) because of the internal way edje_cc compiles the theme (kind of
static link).
By executing an application under the player (option
--external-injection), actions can be remotely forwarded.
The communication is done via Eina Debug channel. Therefore, efl_debugd
must be run before the application.
An injection tool has been implemented to show how to communicate with
the application.
The exu doesn't contain the old scenario entries (Variant_st) but the new
(Exactness_Action).
The recorder doesn't create old rec file but only exu.
The player/inspector support rec files by converting them to the new
format internally.
The structures have been tried to be simpler.
Daniel Zaoui