this is a performance optimization. it brings in a "stat generation".
for now it's disabled by default so we retain previous behavior. this
stops eina file from opening and stating a file every time you open
... it only does it if stat generation is off, or, if the generation
changed since the last time it opened that file. this makes cache hits
not have a 3 syscall cost (open+fstat+close). this optimizes that
lower end of things path. but .. it comes at a cost. if the file
changes before generation ticks over (which this forces to tick over
every time the loop exits idle by default).
now here is something to ask.
1. should we have this on by default and accept the "inexactness"
since you can eina_file_statgen_next() before any call that would do
i/o to force it to look at the real file stat info...
2. should we tick over every idle enter OR every N idle enters or
every frame we render instead? ... i want to avoid getting a timestamp
or having a timer interrupt often... so what should we do?
at least this introduces the idea, some api's and an env var to turn
this on. it definitely cuts down syscalls during things like creation
of widdgets or objects in large batches etc.
on linux open supports() O_CLOEXEC. add test case for this in meson
build and ifdef. this rolls 3 syscalls into 1 as we were doing open,
then fnctl to get and fcntl to set flags.
less syscalls is a good thing as syscalls are not cheap on some
architectures or systems. I've seen a syscall on 1 system take 2-3x
as long as another and another syscall in the same 2 system
comparison take 10x as long. depending on the syscall you may only
have a budget of something like 5000 syscalls "per frame" (60fps)
before you spend all of your frame time just in syscalls not
doing any processing, so we should keep these down if possible
and that is what this does.
Summary:
this function is commonly used to detect the existence of files using
its return value. for this purpose, printing warnings any time the file
cannot be opened is spammy and not very helpful in the context of detecting
actual errors.
Depends on D8950
Reviewers: devilhorns
Reviewed By: devilhorns
Subscribers: cedric, #reviewers, #committers
Tags: #efl
Differential Revision: https://phab.enlightenment.org/D8951
As the filename is now a stringshare, also make sure virtual
files use stringshares for the filename! Also when unmapping
we still need to test whether it is copied or not as unmap
will break on less tolerant architectures.
@fix T6449
Summary:
Unsigned integer should not be compared to less than zero
@fix
Test Plan: NA
Reviewers: cedric
Reviewed By: cedric
Subscribers: jpeg
Differential Revision: https://phab.enlightenment.org/D5269
Signed-off-by: Cedric Bail <cedric@osg.samsung.com>
This makes sure that the call to madvise is safe. On Linux it's not too
much of an issue as checks are made inside madvise, and the worst that
can happen is an error is returned (EINVAL). Not great.
But if MAP_POPULATE is not present, as is the case on *BSD, then the
internal function _eina_file_map_populate() is used for the populate
rule. In that case actual data is read and we should make sure not to
trigger a segfault or bus error.
Also, this makes sure that in case of HugeTLB we actually populate all
pages, rather than one page out of 8 (we were jumping by 16Mb instead of
2Mb).
Note: Can we get the size of a HugeTLB at runtime? We're assuming 2Mb
which might very well not be the case!
See: https://wiki.debian.org/Hugepages
Tested by disabling MAP_POPULATE and observing crashes :)
@fix
restrict mapping /dev/zero to only eina files having a sigbus
reported. the mmap was before all our file access used eina_file i
think thus the raw mmap of it. now walk all eina files and find the
candidate and only then if it exists flag is as having a faulty i/o
backing and map the zerto pages then return, otherwise call abort.
more restricted mapping and perhaps a fix for not trapping non-efl
issues.
@fix
When a virtualized file is created the file->global_map will not
point to a mmapped region, thus it's not safe to use munmap() during
the file cleanup. Only use munmap() if the file is backed by a FD.
Fixes: T5234.
Signed-off-by: Cedric BAIL <cedric@osg.samsung.com>
Summary:
This was causing problems on non-Linux architectures as eina_file_real_close unmapped not mapped data. Added a "copied" flag to Eina_File which is set on eina_file_virtualize (on copied data), and tested for when eina_file_real_close does the unmap. I'm surprised Linux allowed this. Certainly all of the BSDs crashed with the previous behaviour.
@fix T5479
Test Plan: Example inlcude Rage and Enlightenment Thumb on BSD systems which use eina_file_virtualize with emotion to obtain album artwork.
Reviewers: raster, cedric, jpeg
Reviewed By: jpeg
Maniphest Tasks: T5479
Differential Revision: https://phab.enlightenment.org/D5006
So glibc has decided that readdir_r is hard to use safely and deprecated it
this summer. They recommand to use readdir, which was in the past unsafe to
use in a multi thread scenario, but is now on most system (and all system
we care, including our own implementation in evil). It is basically safe
as long the same DIRP is not accessed from another thread. This is true in
our code base, so we are fine to go with this.
For further reading: https://lwn.net/Articles/696474/
Before this patch, we were unconditionnaly destroying the Eina_File if that one
did change on disk. We also make sure that we remove the right entry from the cache
if the file did change there.
The lock on the main hash was taken to late (after we took the decision
to remove the targeted Eina_File from the cache), this means it was possible
to get an Eina_File from the cache that was going to be removed. This patch
attempt to fix that potential race condition.
Hopefully should fix T461.
global_map is set to MAP_FAILED in case of error after mmap.
So, it is initialized to MAP_FAILED and considered valid
otherwise.
So, we don't want to set the map to NULL or even check again NULL.
This reverts commit c002d113f1.
This commit reliably breaks builds with clang. Please test with
export CC=clang before you commit again.
export CC=clang
./autogen.sh --enable-multisense && make -j 10
The error Jenkins gets (I confirmed locally):
/bin/bash: line 1: 13549 Segmentation fault (core dumped) EFL_RUN_IN_TREE=1 ./bin/edje/edje_cc -id . -fd . -id ./tests/emotion/data tests/emotion/data/theme.edc tests/emotion/data/theme.edj
it's useful to copy file from one place to another and this will be
used in eio' s implementation.
NOTE: did not use mmap here as mmap faults may be cumbersome to handle
(Eina_File itself does that, but in a nasty way) and the
implementation would be severely different as there is no Eina_File
from FD, and there is no way to inject custom memory/fd into the
Eina_File's fault handling. The performance would not be that
different anyways and the splice() is already in there for systems
with good performance (read: Linux).
SVN revision: 81942