the alignbment logic was wrong. we have to use the worst case. that
means 8 or 16 byte alignment. eina mempool alignment logic is wrong
for this as it assumes an array of typoes of all the same size...
this fixes crashes seen on armv7 with sigbus in new gesture code which
got unlucky.
@fix
Summary:
integrate mman.h to make Evil private to the EFL, as mman.h does not exist on Windows. After a discussion with raster, i include sys/mman.h only on non Windows platform.
One issue, though, is that src/modules/emotion/generic/Emotion_Generic_Plugin.h has inlined functions using mmap()
Test Plan: compilation on Windows
Reviewers: cedric, raster, zmike
Subscribers: #reviewers, #committers
Tags: #efl
Differential Revision: https://phab.enlightenment.org/D9542
we can't sensibly use things like massif to track memory if we bypass
itr with mmaping -1 fd anonymous memory... so if built with valgrind
support and running under valgrind, use malloc/calloc and free so
these tools actually do something useful for these bits of memory.
luajit wants to check the upper 17 bits of pointers and assume they
are all 0 if on a 64bit architecture. it will panic and barf if they
are used. we have been using them in our eoid's for a long time.
my take on this is that this is unportable. assuming how many bits are
or are not valid in an opaque pointer (void *) that is simply passed
to you and you cannot dereference or use (no size information even for
what amount of data it points to etc.), is just wrong. it's not
portable and it's trying to be too smart and creating such issues. my
take is that luajit needs a fix for this in the longer term. but for
now let's have a 47 bit mode and go with that. it does mean i have to
drop our generation counter to 10 bits on 64bit (from 27 bits) which
increases likelihood of eoid re-use being falsely detected as valid
(before on 64bit it was 1 in 130 million or so chance, with this 47
bit change it's 1 in 1000. but to be fair on 32bit it's 7 bits for gen
count so 1 in 127 ... so still more than 10x "safer" than on 32bit...
but still...). the relevant check in luajit is:
(((uint64_t)(p) >> 47) ? (lj_err_msg(L, LJ_ERR_BADLU), NULL) : (p))
it ONLY does this on 64bit. on 32bit pointers are not checked for
validity at all.
as an aside, armv8.2 seemingly will bring 52bit addresses so luajit is
going to fall over flat on a newer rev of armv8. it may be x86 also
uses more bits. last i knew it was 48bits so the 47 bit check luajit
does i think is even wrong for x86-64. to be detailed i read:
amd64 arch == 48 bits (so luajit is wrong). even better In addition,
the AMD specification requires that the most significant 16 bits of
any virtual address, bits 48 through 63, must be copies of bit 47 (in
a manner akin to sign extension). so if the upper bit of 48 is set
THEN all the 16 upper bits must be 1... breaking luajit, even if it
were 47bit and this rule applied. I read the architecture allows for
up to 52bits of actual addresses so architecture-wise this is even
wrong...
So I smell a core bug here in luajit. Certainly in the number of bits
it insists must be 0 (the upper 17 bits where on amd64/x86-64 it
should be the upper 16 bits... and even then these may NOT be 0 if bit
47 (the upper bit of the lower 48 is 1).... so the whole check is
invalid... :(
at least the above is at a theoretical level. i believe that the
addresses divide the 48 bits into 2 chunks (thus 47)... but at the
PHYSICAL level with no mmu and virtual memory. arm64 has this:
https://www.kernel.org/doc/Documentation/arm64/memory.txt
note in all cases the 2nd chunk of memory has at leats some upper bits
of physical addresses beign 1 ... which makes luajit invalid tyo use
without virtual memory remapping these away from high bits.
@fix
Instead of 2 sets of macro, one for HAVE_EO_ID and another without,
use a single set of macros and have the implementation of
_eo_class_pointer_get() and _eo_obj_pointer_get() to do the actual
These functions now take the source information so the logs reflect
that and not always the same function.
so there were a few issues. one we had a spinlokc on the eoid table
for shared objects AND then had a mutex for accessing those objects
(released on return from any eo function). BUT this missed some funcs
like eo_ref, eo_unref and so on in eo.c ... oops. so fixed. but then i
realized there was a race condition. we locked the eoid table then
unlocked with our pointer THEN locked the sharted object mutex ...
then unlocked it. that was a race condtion gap. so we should share the
same lock anyway - if it's a shared object, grab the shared object
mutex then do a lookup and if the lookup does not fail, KEEP the lock
until it is released by the return from eo function or by some special
macro/funcs that released a matching lock. since its a recursive lock
this is all fine. as its also a universal single lock for all objects
we just need the eoid to know if it's shared and needs locking based
on the domain bits. so now do this locking properly with just a single
mutex, not both a spinlock and mutex and keep the lock around until
totally done with the object. this plugs the race condition holes and
goes from 1 spinlock lock and unlock then a mutex lock and unlokc to
just a single mutex lock and unlock. this means shared objects are
actually truly safe across threads and only have the overhead of a
single recursive mutex to lock and unlock in every api call.
we now just lost another bit from generation count. down to 6 in 32bit
and 26 in 64 bit. this sucks but is necessary. now we are using the
bits just below ref and super bits the code was just maskign off the
next bit as a class marker. this was so so so so wrong. it was the ide
table space. we just never used numbers high enough to start using it.
since i added domain there now those bits can be used easily with
thread domain or other domain. argh! existing eo bug found and fixed.
annoying! :) i added another #define there just to be clear we use
that bit for classes.
this adds a signle mutex (recursive) mutex for all eo objects that is
auto-called by _efl_object_call_resolve() and _efl_object_call_end()
that wrap all eo method calls and since its recursive it can be
blindly called for sub-calls. this will lock all shared objects during
any call to any shared object so only the thread calling now has
access until it releases. not fine-grained but good enough and the
best we can do "simplistically".
This moved all the eoid tables, eoid lookup caches, generation count
information ad eo_isa cache into a TLS segment of memory that is
thread private. There is also a shared domain for EO objects that all
threads can access, but it has an added cost of a lock. This means
objects accessed outside the thread they were created in cannot be
accessed by another thread unless they are adopted in temporarily, or
create4d with the shared domain active at the time of creation. child
objects will use their parent object domain if created with a parent
object passed in. If you were accessing EO (EFL) objects across threads
before then this will actually now cause your code to fail as it was
invalid before to do this as no actual objects were threadsafe in EFL,
so this will force things to "fail early".
ecore_thread_main_loop_begin() and end() still work as this uses the
eo domain adoption features to temporarily adopt a domain during this
section and then return it when done.
This returns speed back to eo brining the overhead in my tests of
lookup for the elm genlist autobounce test in elementary from about
5-7% down to 2.5-2.6%. A steep drop.
This does not mean everything is perfect. Still to do are:
1. Tests in the test suite
2. Some API's to help for sending objects from thread to thread
3. Make the eo call cache TLS data to make it also safe
4. Look at other locks in eo and probably move them to TLS data
5. Make eo resolve and call wrappers that call the real method func do
recursive mutex wrapping of the given object IF it is a shared object
to provide threadsafety transparently for shared objects (but adding
some overhead as a result)
6. Test test est, and that is why this commit is going in now for wider
testing
7. Decide how to make this work with sending IPC (between threads)
8. Deciding what makes an object sendable (a sendable property in base?)
9. Deciding what makes an object shareable (a sharable property in base?)
The syntax is described in: https://phab.enlightenment.org/w/eo/
Summary:
eo_do(obj, a_set(1)) -> a_set(obj, 1)
eo_do_super(obj, CLASS, a_set(1)) -> a_set(eo_super(obj, CLASS), 1)
eo_do_*_ret() set of functions are no longer needed.
This is the first step, the next step would be to also fix up eo_add()
which currently still uses the old syntax and is not 100% portable.
@feature
In some cases, invalid object ids (e.g 0x1) would pass validation and
represent completely different objects (0x80...01). This happened because
we weren't properly checking a given object id is actually an object id.
@fix.
We check _current_table for NULL, and then populate it (it's a global)
through another function, but we don't really check it's not NULL before
using it, we just assume because of an indirect other variable.
This confused coveritiy, can confuse humans too, and in general risky
(if something changes).
CID 1039419
due to recent changes a lot of objects are now NULL (correctly) and eo
complains on access of them. it's simply too noisy adding too many
if's all through code, so let's just make eo be sensible here.
This patch sets the one before most significant bit on for classes. This
means that class ids are now very big, compared to the old ids which
were growing small integers (1, 2, 3...).
This makes accidental passing of integers (corrupted obj pointers) less
common.
@feature
this is the first step on the road to remove class specific EAPI from Eo.h
using this handle we will know if a Eo* is a class or an object pointer
Conflicts:
src/lib/eo/eo.c
Carsten Haitzler
Vincent Torri
Jean-Philippe Andre
Gustavo Sverzut Barbieri
Cedric BAIL
Jaehyun Cho
Tom Hacohen
Daniel Kolesa
Jean Guyomarc'h
Jérémy Zurcher
Cedric Bail