Commit Graph

97 Commits

Author SHA1 Message Date
Daniel Hirt 365630b8db Use correct output dir for reports
Not really sure if it needs to generate reports on 'init' mode, but at least now
it will be in the correct directory.
2016-07-13 16:30:07 +00:00
Daniel Zaoui a21c2f39cc Add an new executable to help on recordings maintenance
An issue that currently happens in Exactness is that we don't have any
way to debug the recordings.
Only debug information can help us to figure out what data is stored
inside the rec files.

Three commands are available:
- Clean: remove bad timestamp events and duplicate events
- Add a delay: because of the first event is directly treated if no
first timestamp is present, we need a way to fix recordings by adding
them a delay before the first event.
- List information: display the list of events, as vieet doesn't work at
all. Really helpful to figure out bugs.
2016-03-27 11:53:44 +03:00
Stefan Schmidt 33787ceb6a Revert "replace "compare" utility with custom compare function using only EFL."
This reverts commit 7989552b29.

This change introduces segfaults on my machine as well as Jenkins every time a
test fails and a comparison of the images is needed. We might want to bring this
back later but for now a working solution is preferred.
2016-03-22 14:13:56 +01:00
Daniel Zaoui 4b2887721b Force scale and finger size
These two environment variables need to be forced to the standard
values to make sure scenarios are well applied.
Without this forcing, mouse clicks are not done at the right
coordinates, leading to a different behavior of the application. In
addition, shots comparison will always fail if the series of images are
based on different values of these variables.
2016-03-02 12:46:26 +02:00
kabeer khan 7989552b29 replace "compare" utility with custom compare function using only EFL.
Summary: Signed-off-by: kabeer khan <>

Reviewers: cedric, tasn

Reviewed By: cedric, tasn

Subscribers: tasn

Differential Revision:

Signed-off-by: Cedric BAIL <>
2015-07-29 23:27:29 +02:00
kabeer khan bb051fce2b Prevent deletion of shots having same prefix as test name
Summary: Signed-off-by: kabeer khan <>

Reviewers: tasn

Differential Revision:
2015-06-09 13:52:53 +01:00
kabeer khan 773e539083 Avoid comparing of tests in play step having common prefix
Currently genlist_group is also running in play step even if it is
commented in tests.txt. Fixed this issue

Signed-off-by: kabeer khan <>

Reviewers: cedric, stefan_schmidt, tasn

Subscribers: stefan_schmidt

Differential Revision:
2015-06-04 10:50:33 +01:00
kabeer khan 834021b743 Enabled exactness to use efl beta API's
Summary: Signed-off-by: kabeer khan <>

Reviewers: cedric, tasn

Differential Revision:
2015-05-27 13:12:40 +01:00
kabeer khan e02ad005f1 Using sha1 to compare two images in play step instead of md5
Using using emile to calculate image sha1 instead of md5 to compare
two images, hence making exactness independant of md5.

Signed-off-by: kabeer khan <>

Reviewers: cedric, tasn

Differential Revision:
2015-05-27 11:41:28 +01:00
kabeer khan c883067d4b Added .gitignore in bin folder to ignore exactness binary
Summary: Signed-off-by: kabeer khan <>

Reviewers: tasn

Differential Revision:
2015-02-26 13:10:01 +00:00
Tom Hacohen 7f40296382 Sort exactness errors. 2013-06-14 13:33:32 +01:00
Tom Hacohen b5523094af Print an error if report file can't be opened for writing. 2013-06-12 15:15:02 +01:00
Tom Hacohen 2d5cc83ed5 Also fail if exit signal is non-zero.
We should fail if we segfault, or exit because of another error that
ecore_exe reports a zero exit code for.
2013-06-10 16:57:48 +01:00
Tom Hacohen ed1d015b8f Added more verbose output to test buildbot. 2013-06-10 13:44:36 +01:00
Tom Hacohen afb0cd3cd1 Added the 'wrap' parameter, to use a wrapping command around the tests.
A good example would be valgrind.
2013-05-21 15:35:35 +01:00
Tom Hacohen c99fdfb239 Added some verbose support (including cursor in simulation). 2013-05-17 15:47:08 +01:00
Tom Hacohen 36fe181d8e Moved recordings to exactness-data-elm. 2013-05-17 11:23:24 +01:00
Tom Hacohen 3e4c93bd83 Make the Exactness error report valid XHTML 1.0 Strict. 2013-05-15 17:20:00 +01:00
Tom Hacohen 11cc575120 Added html output for easier viewing of the errors. 2013-05-15 16:48:04 +01:00
Daniel Willmann feac36f368 Fix destdir usage for destdir != ./
Some more checks would be good, but this fixes basic usage for now.

Signed-off-by: Daniel Willmann <>
2013-05-09 13:59:58 +01:00
Tom Hacohen 93ed31024e Removed accidentally added printf. 2013-05-08 15:48:21 +01:00
Tom Hacohen c5860fc6e6 Fixed exactness compilation. Thanks seoz. 2013-05-08 15:43:19 +01:00
Tom Hacohen c2547c0199 Made the exactness script a C program (with improvements).
It's now considerably faster thanks to concurrency.
It's cleaner (almost the same loc).
Better error output.
Easier to use.
2013-05-08 11:47:59 +01:00
Tom Hacohen f51a7e4835 Exactness: Removed useless bin dir. Merging that into elm_test.
SVN revision: 67934
2012-02-14 13:52:58 +00:00
Aharon Hillel b9406b348a exactness: Cleaned tsuite_evas_hook
TODO: Need to add support of multiple evas for messaging

Signed-off-by: Aharon Hillel <>

SVN revision: 67922
2012-02-14 13:52:22 +00:00
Aharon Hillel 034738063c exactness: Initial version of recording lib.
TODO: Need to refine, clean code. TODO: Later fix tests issues.

Signed-off-by: Aharon Hillel <>

SVN revision: 67921
2012-02-14 13:52:17 +00:00
Aharon Hillel f0ce78d98a exactness: Added support to run user-app - first commit.
TODO: Currently it accepts apps as argument, the problem: We still fail to record or play because before recording I need to have the Evas *e pointer of the app-window to be able to inject events to window evas object. Review my attempts in tsuite.c.evas_new, tsuite_evas_hook.c.evas_new

Signed-off-by: Aharon Hillel <>

SVN revision: 67919
2012-02-14 13:52:10 +00:00
WooHyun Jung c22a29b392 [exactness/test_entry] Change elm_entry_text_filter_append to

SVN revision: 67750
2012-02-08 07:53:33 +00:00
ChunEon Park e91d180b9f exactness - Elm_Genlist_Item -> Elm_Object_Item
SVN revision: 67384
2012-01-20 10:29:35 +00:00
ChunEon Park 24ef320096 exactness - Elm_List_Item -> Elm_Object_Item
SVN revision: 67121
2012-01-12 06:29:09 +00:00
Daniel Juyung Seo 5df77cb4a5 exactness: Applied recent elm toolbar API changes. elm_toolbar_mode_shrink_set -> elm_toolbar_shrink_mode_set
SVN revision: 66942
2012-01-06 11:49:05 +00:00
Daniel Juyung Seo 4e61501e3f exactness: Sync with recent elm. Fixed build break.
SVN revision: 66775
2012-01-03 05:20:38 +00:00
Martin Jansa 140e3018df replace remaining delete-request usages with delete,request which is emited by elm_win, update focus callback names in comments too
Signed-Off-By: Martin Jansa <>

SVN revision: 66729
2012-01-02 05:38:18 +00:00
ChunEon Park fe8c1befc5 exactness - Elm_Diskselector -> Elm_Object_Item
SVN revision: 66678
2011-12-30 09:07:09 +00:00
ChunEon Park 8b880869f5 exactness - Elm_Index_Item -> Elm_Object_Item
+ indentation fix

SVN revision: 66660
2011-12-30 05:02:03 +00:00
Aharon Hillel e0f368a947 exactness: removed redundent tests
Signed-off-by: Aharon Hillel <>

SVN revision: 66391
2011-12-20 13:41:36 +00:00
Aharon Hillel 1be85311bc exactness: Fixed test
Signed-off-by: Aharon Hillel <>

SVN revision: 66390
2011-12-20 13:41:34 +00:00
Aharon Hillel 0d2d5f4510 exactness: Removed redundent (NA) tests
Signed-off-by: Aharon Hillel <>

SVN revision: 66388
2011-12-20 13:41:25 +00:00
Daniel Juyung Seo 70a75738ee exactness: Applied recent elm changes.
SVN revision: 66380
2011-12-20 09:37:52 +00:00
Aharon Hillel 17db9e1237 exactness: Fixed total-tests count
Signed-off-by: Aharon Hillel <>

SVN revision: 66376
2011-12-20 09:09:05 +00:00
Aharon Hillel 164f920c36 exactness: Fixed test errors. TODO: still need to fix two errors in test_index.c, result of API state 3 Signed-off-by: Aharon Hillel <>
SVN revision: 66375
2011-12-20 09:08:53 +00:00
Aharon Hillel 6b7bf9ee9f exactness: Fixed test-handling in case no REC file
Signed-off-by: Aharon Hillel <>

SVN revision: 66240
2011-12-15 09:04:38 +00:00
Aharon Hillel 85ffe5ac15 exactness: Added BaseDir arg,using GETOPT to process command line 1. Added BaseDir argument [-b BaseDir] to tell exactness where to find rec 2. tsuite.c now using ecore getopt to process command line args
Signed-off-by: Aharon Hillel <aharon@aharon-linux-sm.(none)>

SVN revision: 66117
2011-12-12 13:55:05 +00:00
Aharon Hillel b771ef56bc exactness: Added basedir argument for rec.
Signed-off-by: Aharon Hillel <>

SVN revision: 66116
2011-12-12 13:55:02 +00:00
Aharon Hillel e08558e527 exactness: fixed warnings of deprecated funcs
Signed-off-by: Aharon Hillel <>

SVN revision: 65984
2011-12-07 09:23:56 +00:00
Daniel Juyung Seo 897589476e exactness: Fixed build break with latest elementary.

SVN revision: 65979
2011-12-07 00:06:52 +00:00
Aharon Hillel e820a4033d Exactness: initial commit.
(Commit message by TAsn):
Exactness is a pixel perfect test suite for elm/evas/edje.
Exactness lets you write tests, and then record a specific interaction
with them, while taking windowshots in the process. The tests can later
be played back (windowshots will be automatically taken) and the
pictures will be compared for differences (usage of fail_if is also
There is a premade set of tests and recordings for all (most?) of the
elementary widgets in various scenarios.
Because of the nature of this test suite, it doesn't handle well any
tests with continued running animations/viedos (timing can never be 100%
      right). But you can use it to test widgets with transition
animations. Bottom line: just give it a go.

Read the README for more inforamtion.

I hope it'll be deployed on our servers soon, as we really need
automatic testing.

Signed-off-by: Aharon Hillel <>

SVN revision: 65961
2011-12-06 16:05:14 +00:00