diff options
author | Michael Sevakis <jethead71@rockbox.org> | 2013-08-05 22:02:45 -0400 |
---|---|---|
committer | Michael Sevakis <jethead71@rockbox.org> | 2014-08-30 03:48:23 +0200 |
commit | 7d1a47cf13726c95ac46027156cc12dd9da5b855 (patch) | |
tree | eb20d07656806479a8e1fea25887a490ea30d1d8 /firmware/common/dircache.c | |
parent | 95a4c3afcd53a1f8b835dec33de51f9c304de4d9 (diff) | |
download | rockbox-7d1a47cf13726c95ac46027156cc12dd9da5b855.tar.gz rockbox-7d1a47cf13726c95ac46027156cc12dd9da5b855.zip |
Rewrite filesystem code (WIP)
This patch redoes the filesystem code from the FAT driver up to the
clipboard code in onplay.c.
Not every aspect of this is finished therefore it is still "WIP". I
don't wish to do too much at once (haha!). What is left to do is get
dircache back in the sim and find an implementation for the dircache
indicies in the tagcache and playlist code or do something else that
has the same benefit. Leaving these out for now does not make anything
unusable. All the basics are done.
Phone app code should probably get vetted (and app path handling
just plain rewritten as environment expansions); the SDL app and
Android run well.
Main things addressed:
1) Thread safety: There is none right now in the trunk code. Most of
what currently works is luck when multiple threads are involved or
multiple descriptors to the same file are open.
2) POSIX compliance: Many of the functions behave nothing like their
counterparts on a host system. This leads to inconsistent code or very
different behavior from native to hosted. One huge offender was
rename(). Going point by point would fill a book.
3) Actual running RAM usage: Many targets will use less RAM and less
stack space (some more RAM because I upped the number of cache buffers
for large memory). There's very little memory lying fallow in rarely-used
areas (see 'Key core changes' below). Also, all targets may open the same
number of directory streams whereas before those with less than 8MB RAM
were limited to 8, not 12 implying those targets will save slightly
less.
4) Performance: The test_disk plugin shows markedly improved performance,
particularly in the area of (uncached) directory scanning, due partly to
more optimal directory reading and to a better sector cache algorithm.
Uncached times tend to be better while there is a bit of a slowdown in
dircache due to it being a bit heavier of an implementation. It's not
noticeable by a human as far as I can say.
Key core changes:
1) Files and directories share core code and data structures.
2) The filesystem code knows which descriptors refer to same file.
This ensures that changes from one stream are appropriately reflected
in every open descriptor for that file (fileobj_mgr.c).
3) File and directory cache buffers are borrowed from the main sector
cache. This means that when they are not in use by a file, they are not
wasted, but used for the cache. Most of the time, only a few of them
are needed. It also means that adding more file and directory handles
is less expensive. All one must do in ensure a large enough cache to
borrow from.
4) Relative path components are supported and the namespace is unified.
It does not support full relative paths to an implied current directory;
what is does support is use of "." and "..". Adding the former would
not be very difficult. The namespace is unified in the sense that
volumes may be specified several times along with relative parts, e.g.:
"/<0>/foo/../../<1>/bar" :<=> "/<1>/bar".
5) Stack usage is down due to sharing of data, static allocation and
less duplication of strings on the stack. This requires more
serialization than I would like but since the number of threads is
limited to a low number, the tradoff in favor of the stack seems
reasonable.
6) Separates and heirarchicalizes (sic) the SIM and APP filesystem
code. SIM path and volume handling is just like the target. Some
aspects of the APP file code get more straightforward (e.g. no path
hashing is needed).
Dircache:
Deserves its own section. Dircache is new but pays homage to the old.
The old one was not compatible and so it, since it got redone, does
all the stuff it always should have done such as:
1) It may be update and used at any time during the build process.
No longer has one to wait for it to finish building to do basic file
management (create, remove, rename, etc.).
2) It does not need to be either fully scanned or completely disabled;
it can be incomplete (i.e. overfilled, missing paths), still be
of benefit and be correct.
3) Handles mounting and dismounting of individual volumes which means
a full rebuild is not needed just because you pop a new SD card in the
slot. Now, because it reuses its freed entry data, may rebuild only
that volume.
4) Much more fundamental to the file code. When it is built, it is
the keeper of the master file list whether enabled or not ("disabled"
is just a state of the cache). Its must always to ready to be started
and bind all streams opened prior to being enabled.
5) Maintains any short filenames in OEM format which means that it does
not need to be rebuilt when changing the default codepage.
Miscellaneous Compatibility:
1) Update any other code that would otherwise not work such as the
hotswap mounting code in various card drivers.
2) File management: Clipboard needed updating because of the behavioral
changes. Still needs a little more work on some finer points.
3) Remove now-obsolete functionality such as the mutex's "no preempt"
flag (which was only for the prior FAT driver).
4) struct dirinfo uses time_t rather than raw FAT directory entry
time fields. I plan to follow up on genericizing everything there
(i.e. no FAT attributes).
5) unicode.c needed some redoing so that the file code does not try
try to load codepages during a scan, which is actually a problem with
the current code. The default codepage, if any is required, is now
kept in RAM separarately (bufalloced) from codepages specified to
iso_decode() (which must not be bufalloced because the conversion
may be done by playback threads).
Brings with it some additional reusable core code:
1) Revised file functions: Reusable code that does things such as
safe path concatenation and parsing without buffer limitations or
data duplication. Variants that copy or alter the input path may be
based off these.
To do:
1) Put dircache functionality back in the sim. Treating it internally
as a different kind of file system seems the best approach at this
time.
2) Restore use of dircache indexes in the playlist and database or
something effectively the same. Since the cache doesn't have to be
complete in order to be used, not getting a hit on the cache doesn't
unambiguously say if the path exists or not.
Change-Id: Ia30f3082a136253e3a0eae0784e3091d138915c8
Reviewed-on: http://gerrit.rockbox.org/566
Reviewed-by: Michael Sevakis <jethead71@rockbox.org>
Tested: Michael Sevakis <jethead71@rockbox.org>
Diffstat (limited to 'firmware/common/dircache.c')
-rw-r--r-- | firmware/common/dircache.c | 3981 |
1 files changed, 2763 insertions, 1218 deletions
diff --git a/firmware/common/dircache.c b/firmware/common/dircache.c index b53fc4d7a6..1e580bf3af 100644 --- a/firmware/common/dircache.c +++ b/firmware/common/dircache.c | |||
@@ -8,6 +8,7 @@ | |||
8 | * $Id$ | 8 | * $Id$ |
9 | * | 9 | * |
10 | * Copyright (C) 2005 by Miika Pekkarinen | 10 | * Copyright (C) 2005 by Miika Pekkarinen |
11 | * Copyright (C) 2014 by Michael Sevakis | ||
11 | * | 12 | * |
12 | * This program is free software; you can redistribute it and/or | 13 | * This program is free software; you can redistribute it and/or |
13 | * modify it under the terms of the GNU General Public License | 14 | * modify it under the terms of the GNU General Public License |
@@ -18,13 +19,7 @@ | |||
18 | * KIND, either express or implied. | 19 | * KIND, either express or implied. |
19 | * | 20 | * |
20 | ****************************************************************************/ | 21 | ****************************************************************************/ |
21 | |||
22 | /* TODO: | ||
23 | - Allow cache live updating while transparent rebuild is running. | ||
24 | */ | ||
25 | |||
26 | #include "config.h" | 22 | #include "config.h" |
27 | |||
28 | #include <stdio.h> | 23 | #include <stdio.h> |
29 | #include <errno.h> | 24 | #include <errno.h> |
30 | #include "string-extra.h" | 25 | #include "string-extra.h" |
@@ -33,6 +28,8 @@ | |||
33 | #include "debug.h" | 28 | #include "debug.h" |
34 | #include "system.h" | 29 | #include "system.h" |
35 | #include "logf.h" | 30 | #include "logf.h" |
31 | #include "fileobj_mgr.h" | ||
32 | #include "pathfuncs.h" | ||
36 | #include "dircache.h" | 33 | #include "dircache.h" |
37 | #include "thread.h" | 34 | #include "thread.h" |
38 | #include "kernel.h" | 35 | #include "kernel.h" |
@@ -42,393 +39,1626 @@ | |||
42 | #include "dir.h" | 39 | #include "dir.h" |
43 | #include "storage.h" | 40 | #include "storage.h" |
44 | #include "audio.h" | 41 | #include "audio.h" |
45 | #if CONFIG_RTC | ||
46 | #include "time.h" | ||
47 | #include "timefuncs.h" | ||
48 | #endif | ||
49 | #include "rbpaths.h" | 42 | #include "rbpaths.h" |
43 | #include "linked_list.h" | ||
44 | #ifdef HAVE_EEPROM_SETTINGS | ||
45 | #include "crc32.h" | ||
46 | #endif | ||
50 | 47 | ||
48 | /** | ||
49 | * Cache memory layout: | ||
50 | * x - array of struct dircache_entry | ||
51 | * r - reserved buffer | ||
52 | * d - name buffer for the name entry of the struct dircache_entry | ||
53 | * 0 - zero bytes to assist free name block sentinel scanning (not 0xfe or 0xff) | ||
54 | * |xxxxxx|rrrrrrrrr|0|dddddd|0| | ||
55 | * | ||
56 | * Subsequent x are allocated from the front, d are allocated from the back, | ||
57 | * using the reserve buffer for entries added after initial scan. | ||
58 | * | ||
59 | * After a while the cache may look like: | ||
60 | * |xxxxxxxx|rrrrr|0|dddddddd|0| | ||
61 | * | ||
62 | * After a reboot, the reserve buffer is restored in it's size, so that the | ||
63 | * total allocation size grows: | ||
64 | * |xxxxxxxx|rrrrrrrrr|0|dddddddd|0| | ||
65 | * | ||
66 | * | ||
67 | * Cache structure: | ||
68 | * Format is memory position independent and uses only indexes as links. The | ||
69 | * buffer pointers are offset back by one entry to make the array 1-based so | ||
70 | * that an index of 0 may be considered an analog of a NULL pointer. | ||
71 | * | ||
72 | * Entry elements are linked together analagously to the filesystem directory | ||
73 | * structure with minor variations that are helpful to the cache's algorithms. | ||
74 | * Each volume has a special root structure in the dircache structure, not an | ||
75 | * entry in the cache, comprising a forest of volume trees which facilitates | ||
76 | * mounting or dismounting of specified volumes on the fly. | ||
77 | * | ||
78 | * Indexes identifying a volume are computed as: index = -volume - 1 | ||
79 | * Returning the volume from these indexes is thus: volume = -index - 1 | ||
80 | * Such indexes are used in root binding and as the 'up' index for an entry | ||
81 | * who's parent is the root directory. | ||
82 | * | ||
83 | * Open files list: | ||
84 | * When dircache is made it is the maintainer of the main volume open files | ||
85 | * lists, even when it is off. Any files open before dircache is enabled or | ||
86 | * initialized must be bound to cache entries by the scan and build operation. | ||
87 | * It maintains these lists in a special way. | ||
88 | * | ||
89 | * Queued (unresolved) bindings are at the back and resolved at the front. | ||
90 | * A pointer to the first of each kind of binding is provided to skip to help | ||
91 | * iterating one sublist or another. | ||
92 | * | ||
93 | * r0->r1->r2->q0->q1->q2->NULL | ||
94 | * ^resolved0 ^queued0 | ||
95 | */ | ||
51 | 96 | ||
52 | /* Queue commands. */ | 97 | #ifdef DIRCACHE_NATIVE |
53 | #define DIRCACHE_BUILD 1 | 98 | #define dircache_lock() file_internal_lock_WRITER() |
54 | #define DIRCACHE_STOP 2 | 99 | #define dircache_unlock() file_internal_unlock_WRITER() |
55 | 100 | ||
56 | #if (MEMORYSIZE > 8) | 101 | /* scan and build parameter data */ |
57 | #define MAX_OPEN_DIRS 12 | 102 | struct sab_component; |
58 | #else | 103 | struct sab |
59 | #define MAX_OPEN_DIRS 8 | 104 | { |
60 | #endif | 105 | struct filestr_base stream; /* scan directory stream */ |
61 | static DIR_CACHED opendirs[MAX_OPEN_DIRS]; | 106 | struct file_base_info info; /* scanned entry info */ |
62 | static char opendir_dnames[MAX_OPEN_DIRS][MAX_PATH]; | 107 | bool volatile quit; /* halt all scanning */ |
108 | struct sab_component *stackend; /* end of stack pointer */ | ||
109 | struct sab_component *top; /* current top of stack */ | ||
110 | struct sab_component | ||
111 | { | ||
112 | int volatile idx; /* cache index of directory */ | ||
113 | int *downp; /* pointer to ce->down */ | ||
114 | int *volatile prevp; /* previous item accessed */ | ||
115 | } stack[]; /* "recursion" stack */ | ||
116 | }; | ||
117 | |||
118 | #else /* !DIRCACHE_NATIVE */ | ||
119 | |||
120 | #error need locking scheme | ||
121 | #define FILESYS_WRITER_LOCK() | ||
122 | #define FILESYS_WRITER_UNLOCK() | ||
63 | 123 | ||
64 | #define MAX_PENDING_BINDINGS 2 | 124 | struct sab_component |
65 | struct fdbind_queue { | 125 | { |
126 | }; | ||
127 | |||
128 | struct sab | ||
129 | { | ||
130 | #ifdef HAVE_MUTLIVOLUME | ||
131 | int volume; | ||
132 | #endif /* HAVE_MULTIVOLUME */ | ||
66 | char path[MAX_PATH]; | 133 | char path[MAX_PATH]; |
67 | int fd; | 134 | unsigned int append; |
135 | }; | ||
136 | #endif /* DIRCACHE_NATIVE */ | ||
137 | |||
138 | enum | ||
139 | { | ||
140 | FRONTIER_SETTLED = 0x0, /* dir entry contents are complete */ | ||
141 | FRONTIER_NEW = 0x1, /* dir entry contents are in-progress */ | ||
142 | FRONTIER_ZONED = 0x2, /* frontier entry permanent mark (very sticky!) */ | ||
143 | FRONTIER_RENEW = 0x4, /* override FRONTIER_ZONED sticky (not stored) */ | ||
68 | }; | 144 | }; |
69 | 145 | ||
70 | /* Unions with char to make pointer arithmetic simpler and avoid casting */ | 146 | enum |
71 | struct dircache_entry { | 147 | { |
72 | struct dirinfo info; | 148 | DCM_BUILD, /* build a volume */ |
149 | DCM_PROCEED, /* merged DCM_BUILD messages */ | ||
150 | DCM_FIRST = DCM_BUILD, | ||
151 | DCM_LAST = DCM_PROCEED, | ||
152 | }; | ||
153 | |||
154 | #define MAX_TINYNAME sizeof (uint32_t) | ||
155 | #define DC_MAX_NAME MIN(MAX_NAME, UINT8_MAX) | ||
156 | |||
157 | /* Throw some warnings if about the limits if things may not work */ | ||
158 | #if MAX_NAME > UINT8_MAX | ||
159 | #warning Need more than 8 bits in name length bitfield | ||
160 | #endif | ||
161 | |||
162 | #if DIRCACHE_LIMIT > ((1 << 24)-255) | ||
163 | #warning Names may not be addressable with 24 bits | ||
164 | #endif | ||
165 | |||
166 | /* data structure used by cache entries */ | ||
167 | struct dircache_entry | ||
168 | { | ||
169 | int next; /* next at same level */ | ||
73 | union { | 170 | union { |
74 | struct dircache_entry *next; | 171 | int down; /* first at child level (if directory) */ |
75 | char* next_char; | 172 | file_size_t filesize; /* size of file in bytes (if file) */ |
76 | }; | 173 | }; |
174 | int up; /* parent index (-volume-1 if root) */ | ||
77 | union { | 175 | union { |
78 | struct dircache_entry *up; | 176 | struct { |
79 | char* up_char; | 177 | uint32_t name : 24; /* indirect storage (.tinyname == 0) */ |
178 | uint32_t length : 8; /* length of name indexed by 'name' */ | ||
80 | }; | 179 | }; |
81 | union { | 180 | unsigned char namebuf[MAX_TINYNAME]; /* direct storage (.tinyname == 1) */ |
82 | struct dircache_entry *down; | ||
83 | char* down_char; | ||
84 | }; | 181 | }; |
85 | long startcluster; | 182 | uint32_t direntry : 16; /* entry # in parent - max 0xffff */ |
86 | char *d_name; | 183 | uint32_t direntries : 5; /* # of entries used - max 21 */ |
87 | }; | 184 | uint32_t tinyname : 1; /* if == 1, name fits in .namebuf */ |
88 | 185 | uint32_t frontier : 2; /* (FRONTIER_* bitflags) */ | |
89 | /* Cache Layout: | 186 | uint32_t attr : 8; /* entry file attributes */ |
90 | * | 187 | #ifdef DIRCACHE_NATIVE |
91 | * x - array of struct dircache_entry | 188 | long firstcluster; /* first file cluster - max 0x0ffffff4 */ |
92 | * r - reserved buffer | 189 | uint16_t wrtdate; /* FAT write date */ |
93 | * d - name buffer for the d_name entry of the struct dircache_entry | 190 | uint16_t wrttime; /* FAT write time */ |
94 | * |xxxxxx|rrrrrrrrr|dddddd| | 191 | #else |
95 | * | 192 | time_t mtime; /* file last-modified time */ |
96 | * subsequent x are allocated from the front, d are allocated from the back, | ||
97 | * using the reserve buffer for entries added after initial scan | ||
98 | * | ||
99 | * after a while the cache may look like: | ||
100 | * |xxxxxxxx|rrrrr|dddddddd| | ||
101 | * | ||
102 | * after a reboot, the reserve buffer is restored in it's size, so that the | ||
103 | * total allocation size grows | ||
104 | * |xxxxxxxx|rrrrrrrrr|dddddddd| | ||
105 | */ | ||
106 | /* this points to the beginnging of the buffer and the first entry */ | ||
107 | static struct dircache_entry *dircache_root; | ||
108 | /* these point to the start and end of the name buffer (d above) */ | ||
109 | static char *d_names_start, *d_names_end; | ||
110 | /* put "." and ".." into the d_names buffer to enable easy pointer logic */ | ||
111 | static char *dot, *dotdot; | ||
112 | #ifdef HAVE_MULTIVOLUME | ||
113 | static struct dircache_entry *append_position; | ||
114 | #endif | 193 | #endif |
194 | dc_serial_t serialnum; /* entry serial number */ | ||
195 | }; | ||
115 | 196 | ||
116 | static DIR_CACHED opendirs[MAX_OPEN_DIRS]; | 197 | /* spare us some tedium */ |
117 | static struct dircache_entry *fd_bindings[MAX_OPEN_FILES]; | 198 | #define ENTRYSIZE (sizeof (struct dircache_entry)) |
118 | |||
119 | static bool dircache_initialized = false; | ||
120 | static bool dircache_initializing = false; | ||
121 | static bool thread_enabled = false; | ||
122 | static unsigned long allocated_size = 0; | ||
123 | static unsigned long dircache_size = 0; | ||
124 | static unsigned long entry_count = 0; | ||
125 | static unsigned long reserve_used = 0; | ||
126 | static unsigned int cache_build_ticks = 0; | ||
127 | static unsigned long appflags = 0; | ||
128 | 199 | ||
200 | /* thread and kernel stuff */ | ||
129 | static struct event_queue dircache_queue SHAREDBSS_ATTR; | 201 | static struct event_queue dircache_queue SHAREDBSS_ATTR; |
130 | static long dircache_stack[(DEFAULT_STACK_SIZE + 0x400)/sizeof(long)]; | 202 | static uintptr_t dircache_stack[DIRCACHE_STACK_SIZE / sizeof (uintptr_t)]; |
131 | static const char dircache_thread_name[] = "dircache"; | 203 | static const char dircache_thread_name[] = "dircache"; |
132 | 204 | ||
133 | static struct fdbind_queue fdbind_cache[MAX_PENDING_BINDINGS]; | 205 | /* struct that is both used during run time and for persistent storage */ |
134 | static int fdbind_idx = 0; | 206 | static struct dircache |
207 | { | ||
208 | /* cache-wide data */ | ||
209 | int free_list; /* first index of free entry list */ | ||
210 | size_t size; /* total size of data (including holes) */ | ||
211 | size_t sizeused; /* bytes of .size bytes actually used */ | ||
212 | union { | ||
213 | unsigned int numentries; /* entry count (including holes) */ | ||
214 | #ifdef HAVE_EEPROM_SETTINGS | ||
215 | size_t sizeentries; /* used when persisting */ | ||
216 | #endif | ||
217 | }; | ||
218 | int names; /* index of first name in name block */ | ||
219 | size_t sizenames; /* size of all names (including holes) */ | ||
220 | size_t namesfree; /* amount of wasted name space */ | ||
221 | int nextnamefree; /* hint of next free name in buffer */ | ||
222 | /* per-volume data */ | ||
223 | struct dircache_volume /* per volume cache data */ | ||
224 | { | ||
225 | uint32_t status : 2; /* cache status of this volume */ | ||
226 | uint32_t frontier : 2; /* (FRONTIER_* bitflags) */ | ||
227 | dc_serial_t serialnum; /* dircache serial number of root */ | ||
228 | int root_down; /* index of first entry of volume root */ | ||
229 | union { | ||
230 | long start_tick; /* when did scan start (scanning) */ | ||
231 | long build_ticks; /* how long to build volume? (ready) */ | ||
232 | }; | ||
233 | } dcvol[NUM_VOLUMES]; | ||
234 | /* these remain unchanged between cache resets */ | ||
235 | size_t last_size; /* last reported size at boot */ | ||
236 | size_t reserve_used; /* reserved used at last build */ | ||
237 | dc_serial_t last_serialnum; /* last serialnumber generated */ | ||
238 | } dircache; | ||
239 | |||
240 | /* struct that is used only for the cache in main memory */ | ||
241 | struct dircache_runinfo | ||
242 | { | ||
243 | /* cache setting and build info */ | ||
244 | int suspended; /* dircache suspend count */ | ||
245 | bool enabled; /* dircache master enable switch */ | ||
246 | unsigned int thread_id; /* current/last thread id */ | ||
247 | bool thread_done; /* thread has exited */ | ||
248 | /* cache buffer info */ | ||
249 | int handle; /* buflib buffer handle */ | ||
250 | size_t bufsize; /* size of buflib allocation - 1 */ | ||
251 | int buflocked; /* don't move due to other allocs */ | ||
252 | union { | ||
253 | void *p; /* address of buffer - ENTRYSIZE */ | ||
254 | struct dircache_entry *pentry; /* alias of .p to assist entry resolution */ | ||
255 | unsigned char *pname; /* alias of .p to assist name resolution */ | ||
256 | }; | ||
257 | struct buflib_callbacks ops; /* buflib ops callbacks */ | ||
258 | /* per-volume data */ | ||
259 | struct dircache_runinfo_volume | ||
260 | { | ||
261 | struct file_base_binding *resolved0; /* first resolved binding in list */ | ||
262 | struct file_base_binding *queued0; /* first queued binding in list */ | ||
263 | struct sab *sabp; /* if building, struct sab in use */ | ||
264 | } dcrivol[NUM_VOLUMES]; | ||
265 | } dircache_runinfo; | ||
266 | |||
267 | #define BINDING_NEXT(bindp) \ | ||
268 | ((struct file_base_binding *)(bindp)->node.next) | ||
269 | |||
270 | #define FOR_EACH_BINDING(start, p) \ | ||
271 | for (struct file_base_binding *p = (start); p; p = BINDING_NEXT(p)) | ||
272 | |||
273 | #define FOR_EACH_CACHE_ENTRY(ce) \ | ||
274 | for (struct dircache_entry *ce = &dircache_runinfo.pentry[1], \ | ||
275 | *_ceend = ce + dircache.numentries; \ | ||
276 | ce < _ceend; ce++) if (ce->serialnum) | ||
277 | |||
278 | #define FOR_EACH_SAB_COMP(sabp, p) \ | ||
279 | for (struct sab_component *p = (sabp)->top; p < (sabp)->stackend; p++) | ||
280 | |||
281 | /* "overloaded" macros to get volume structures */ | ||
282 | #define DCVOL_i(i) (&dircache.dcvol[i]) | ||
283 | #define DCVOL_volume(volume) (&dircache.dcvol[volume]) | ||
284 | #define DCVOL_infop(infop) (&dircache.dcvol[BASEINFO_VOL(infop)]) | ||
285 | #define DCVOL_dirinfop(dirinfop) (&dircache.dcvol[BASEINFO_VOL(dirinfop)]) | ||
286 | #define DCVOL(x) DCVOL_##x(x) | ||
287 | |||
288 | #define DCRIVOL_i(i) (&dircache_runinfo.dcrivol[i]) | ||
289 | #define DCRIVOL_infop(infop) (&dircache_runinfo.dcrivol[BASEINFO_VOL(infop)]) | ||
290 | #define DCRIVOL_bindp(bindp) (&dircache_runinfo.dcrivol[BASEBINDING_VOL(bindp)]) | ||
291 | #define DCRIVOL(x) DCRIVOL_##x(x) | ||
292 | |||
293 | /* reserve over 75% full? */ | ||
294 | #define DIRCACHE_STUFFED(reserve_used) \ | ||
295 | ((reserve_used) > 3*DIRCACHE_RESERVE / 4) | ||
135 | 296 | ||
136 | /* --- Internal cache structure control functions --- */ | 297 | #ifdef HAVE_EEPROM_SETTINGS |
298 | /** | ||
299 | * remove the snapshot file | ||
300 | */ | ||
301 | static int remove_dircache_file(void) | ||
302 | { | ||
303 | return remove(DIRCACHE_FILE); | ||
304 | } | ||
305 | |||
306 | /** | ||
307 | * open or create the snapshot file | ||
308 | */ | ||
309 | static int open_dircache_file(int oflag) | ||
310 | { | ||
311 | return open(DIRCACHE_FILE, oflag, 0666); | ||
312 | } | ||
313 | #endif /* HAVE_EEPROM_SETTINGS */ | ||
137 | 314 | ||
138 | static inline struct dircache_entry* get_entry(int id) | 315 | #ifdef DIRCACHE_DUMPSTER |
316 | /** | ||
317 | * clean up the memory allocation to make viewing in a hex editor more friendly | ||
318 | * and highlight problems | ||
319 | */ | ||
320 | static inline void dumpster_clean_buffer(void *p, size_t size) | ||
139 | { | 321 | { |
140 | return &dircache_root[id]; | 322 | memset(p, 0xAA, size); |
141 | } | 323 | } |
324 | #endif /* DIRCACHE_DUMPSTER */ | ||
142 | 325 | ||
143 | /* flag to make sure buffer doesn't move due to other allocs. | 326 | /** |
144 | * this is set to true completely during dircache build */ | 327 | * relocate the cache when the buffer has moved |
145 | static bool dont_move = false; | 328 | */ |
146 | static int dircache_handle; | 329 | static int move_callback(int handle, void *current, void *new) |
147 | static int move_callback(int handle, void* current, void* new) | ||
148 | { | 330 | { |
149 | (void)handle; | 331 | if (dircache_runinfo.buflocked) |
150 | if (dont_move) | ||
151 | return BUFLIB_CB_CANNOT_MOVE; | 332 | return BUFLIB_CB_CANNOT_MOVE; |
152 | 333 | ||
153 | #define UPDATE(x) if (x) { x = PTR_ADD(x, diff); } | 334 | dircache_runinfo.p = new - ENTRYSIZE; |
154 | /* relocate the cache */ | 335 | |
155 | ptrdiff_t diff = new - current; | 336 | return BUFLIB_CB_OK; |
156 | for(unsigned i = 0; i < entry_count; i++) | 337 | (void)handle; (void)current; |
338 | } | ||
339 | |||
340 | /** | ||
341 | * add a "don't move" lock count | ||
342 | */ | ||
343 | static inline void buffer_lock(void) | ||
344 | { | ||
345 | dircache_runinfo.buflocked++; | ||
346 | } | ||
347 | |||
348 | /** | ||
349 | * remove a "don't move" lock count | ||
350 | */ | ||
351 | static inline void buffer_unlock(void) | ||
352 | { | ||
353 | dircache_runinfo.buflocked--; | ||
354 | } | ||
355 | |||
356 | |||
357 | /** Open file bindings management **/ | ||
358 | |||
359 | /* compare the basic file information and return 'true' if they are logically | ||
360 | equivalent or the same item, else return 'false' if not */ | ||
361 | static inline bool binding_compare(const struct file_base_info *infop1, | ||
362 | const struct file_base_info *infop2) | ||
363 | { | ||
364 | #ifdef DIRCACHE_NATIVE | ||
365 | return fat_file_is_same(&infop1->fatfile, &infop2->fatfile); | ||
366 | #else | ||
367 | #error hey watch it! | ||
368 | #endif | ||
369 | } | ||
370 | |||
371 | /** | ||
372 | * bind a file to the cache; "queued" or "resolved" depending upon whether or | ||
373 | * not it has entry information | ||
374 | */ | ||
375 | static void binding_open(struct file_base_binding *bindp) | ||
376 | { | ||
377 | struct dircache_runinfo_volume *dcrivolp = DCRIVOL(bindp); | ||
378 | if (bindp->info.dcfile.serialnum) | ||
157 | { | 379 | { |
158 | UPDATE(dircache_root[i].d_name); | 380 | /* already resolved */ |
159 | UPDATE(dircache_root[i].next_char); | 381 | dcrivolp->resolved0 = bindp; |
160 | UPDATE(dircache_root[i].up_char); | 382 | file_binding_insert_first(bindp); |
161 | UPDATE(dircache_root[i].down_char); | ||
162 | } | 383 | } |
163 | dircache_root = new; | 384 | else |
164 | UPDATE(d_names_start); | 385 | { |
165 | UPDATE(d_names_end); | 386 | if (dcrivolp->queued0 == NULL) |
166 | UPDATE(dot); | 387 | dcrivolp->queued0 = bindp; |
167 | UPDATE(dotdot); | ||
168 | 388 | ||
169 | for(unsigned i = 0; i < MAX_OPEN_FILES; i++) | 389 | file_binding_insert_last(bindp); |
170 | UPDATE(fd_bindings[i]); | 390 | } |
391 | } | ||
171 | 392 | ||
172 | #ifdef HAVE_MULTIVOLUME | 393 | /** |
173 | UPDATE(append_position); | 394 | * remove a binding from the cache |
174 | #endif | 395 | */ |
175 | return BUFLIB_CB_OK; | 396 | static void binding_close(struct file_base_binding *bindp) |
397 | { | ||
398 | struct dircache_runinfo_volume *dcrivolp = DCRIVOL(bindp); | ||
399 | |||
400 | if (bindp == dcrivolp->queued0) | ||
401 | dcrivolp->queued0 = BINDING_NEXT(bindp); | ||
402 | else if (bindp == dcrivolp->resolved0) | ||
403 | { | ||
404 | struct file_base_binding *nextp = BINDING_NEXT(bindp); | ||
405 | dcrivolp->resolved0 = (nextp == dcrivolp->queued0) ? NULL : nextp; | ||
406 | } | ||
407 | |||
408 | file_binding_remove(bindp); | ||
409 | /* no need to reset it */ | ||
176 | } | 410 | } |
177 | 411 | ||
178 | static struct buflib_callbacks ops = { | 412 | /** |
179 | .move_callback = move_callback, | 413 | * resolve a queued binding with the information from the given source file |
180 | .shrink_callback = NULL, | 414 | */ |
181 | }; | 415 | static void binding_resolve(const struct file_base_info *infop) |
416 | { | ||
417 | struct dircache_runinfo_volume *dcrivolp = DCRIVOL(infop); | ||
418 | |||
419 | /* quickly check the queued list to see if it's there */ | ||
420 | struct file_base_binding *prevp = NULL; | ||
421 | FOR_EACH_BINDING(dcrivolp->queued0, p) | ||
422 | { | ||
423 | if (!binding_compare(infop, &p->info)) | ||
424 | { | ||
425 | prevp = p; | ||
426 | continue; | ||
427 | } | ||
428 | |||
429 | if (p == dcrivolp->queued0) | ||
430 | dcrivolp->queued0 = BINDING_NEXT(p); | ||
431 | else | ||
432 | { | ||
433 | file_binding_remove_next(prevp, p); | ||
434 | file_binding_insert_first(p); | ||
435 | } | ||
436 | |||
437 | dcrivolp->resolved0 = p; | ||
438 | |||
439 | /* srcinfop may be the actual one */ | ||
440 | if (&p->info != infop) | ||
441 | p->info.dcfile = infop->dcfile; | ||
442 | |||
443 | break; | ||
444 | } | ||
445 | } | ||
182 | 446 | ||
183 | #ifdef HAVE_EEPROM_SETTINGS | ||
184 | /** | 447 | /** |
185 | * Open the dircache file to save a snapshot on disk | 448 | * dissolve a resolved binding on its volume |
186 | */ | 449 | */ |
187 | static int open_dircache_file(unsigned flags, int permissions) | 450 | static void binding_dissolve(struct file_base_binding *prevp, |
451 | struct file_base_binding *bindp) | ||
188 | { | 452 | { |
189 | if (permissions != 0) | 453 | struct dircache_runinfo_volume *dcrivolp = DCRIVOL(bindp); |
190 | return open(DIRCACHE_FILE, flags, permissions); | ||
191 | 454 | ||
192 | return open(DIRCACHE_FILE, flags); | 455 | if (bindp == dcrivolp->resolved0) |
456 | { | ||
457 | struct file_base_binding *nextp = BINDING_NEXT(bindp); | ||
458 | dcrivolp->resolved0 = (nextp == dcrivolp->queued0) ? NULL : nextp; | ||
459 | } | ||
460 | |||
461 | if (dcrivolp->queued0 == NULL) | ||
462 | dcrivolp->queued0 = bindp; | ||
463 | |||
464 | file_binding_remove_next(prevp, bindp); | ||
465 | file_binding_insert_last(bindp); | ||
466 | |||
467 | dircache_dcfile_init(&bindp->info.dcfile); | ||
193 | } | 468 | } |
194 | 469 | ||
195 | /** | 470 | /** |
196 | * Remove the snapshot file | 471 | * dissolve all resolved bindings on a given volume |
197 | */ | 472 | */ |
198 | static int remove_dircache_file(void) | 473 | static void binding_dissolve_volume(struct dircache_runinfo_volume *dcrivolp) |
199 | { | 474 | { |
200 | return remove(DIRCACHE_FILE); | 475 | if (!dcrivolp->resolved0) |
476 | return; | ||
477 | |||
478 | FOR_EACH_BINDING(dcrivolp->resolved0, p) | ||
479 | { | ||
480 | if (p == dcrivolp->queued0) | ||
481 | break; | ||
482 | |||
483 | dircache_dcfile_init(&p->info.dcfile); | ||
484 | } | ||
485 | |||
486 | dcrivolp->queued0 = dcrivolp->resolved0; | ||
487 | dcrivolp->resolved0 = NULL; | ||
201 | } | 488 | } |
202 | #endif | 489 | |
203 | /** | 490 | |
204 | * Internal function to allocate a new dircache_entry from memory. | 491 | /** Dircache buffer management **/ |
492 | |||
493 | /** | ||
494 | * allocate the cache's memory block | ||
205 | */ | 495 | */ |
206 | static struct dircache_entry* allocate_entry(void) | 496 | static int alloc_cache(size_t size) |
207 | { | 497 | { |
208 | struct dircache_entry *next_entry; | 498 | /* pad with one extra-- see alloc_name() and free_name() */ |
209 | 499 | return core_alloc_ex("dircache", size + 1, &dircache_runinfo.ops); | |
210 | if (dircache_size > allocated_size - MAX_PATH*2) | 500 | } |
501 | |||
502 | /** | ||
503 | * put the allocation in dircache control | ||
504 | */ | ||
505 | static void set_buffer(int handle, size_t size) | ||
506 | { | ||
507 | void *p = core_get_data(handle); | ||
508 | |||
509 | #ifdef DIRCACHE_DUMPSTER | ||
510 | dumpster_clean_buffer(p, size); | ||
511 | #endif /* DIRCACHE_DUMPSTER */ | ||
512 | |||
513 | /* set it up as a 1-based array */ | ||
514 | dircache_runinfo.p = p - ENTRYSIZE; | ||
515 | |||
516 | if (dircache_runinfo.handle != handle) | ||
211 | { | 517 | { |
212 | logf("size limit reached"); | 518 | /* new buffer */ |
213 | return NULL; | 519 | dircache_runinfo.handle = handle; |
520 | dircache_runinfo.bufsize = size; | ||
521 | dircache.names = size + ENTRYSIZE; | ||
522 | dircache_runinfo.pname[dircache.names - 1] = 0; | ||
523 | dircache_runinfo.pname[dircache.names ] = 0; | ||
214 | } | 524 | } |
215 | 525 | } | |
216 | next_entry = &dircache_root[entry_count++]; | ||
217 | next_entry->d_name = NULL; | ||
218 | next_entry->up = NULL; | ||
219 | next_entry->down = NULL; | ||
220 | next_entry->next = NULL; | ||
221 | 526 | ||
222 | dircache_size += sizeof(struct dircache_entry); | 527 | /** |
528 | * remove the allocation from dircache control and return the handle | ||
529 | */ | ||
530 | static int reset_buffer(void) | ||
531 | { | ||
532 | int handle = dircache_runinfo.handle; | ||
533 | if (handle > 0) | ||
534 | { | ||
535 | /* don't mind .p; it might get changed by the callback even after | ||
536 | this call; buffer presence is determined by the following: */ | ||
537 | dircache_runinfo.handle = 0; | ||
538 | dircache_runinfo.bufsize = 0; | ||
539 | } | ||
540 | |||
541 | return handle; | ||
542 | } | ||
543 | |||
544 | /** | ||
545 | * return the number of bytes remaining in the buffer | ||
546 | */ | ||
547 | static size_t dircache_buf_remaining(void) | ||
548 | { | ||
549 | if (!dircache_runinfo.handle) | ||
550 | return 0; | ||
551 | |||
552 | return dircache_runinfo.bufsize - dircache.size; | ||
553 | } | ||
554 | |||
555 | /** | ||
556 | * return the amount of reserve space used | ||
557 | */ | ||
558 | static size_t reserve_buf_used(void) | ||
559 | { | ||
560 | size_t remaining = dircache_buf_remaining(); | ||
561 | return (remaining < DIRCACHE_RESERVE) ? | ||
562 | DIRCACHE_RESERVE - remaining : 0; | ||
563 | } | ||
564 | |||
565 | |||
566 | /** Internal cache structure control functions **/ | ||
567 | |||
568 | /** | ||
569 | * generate the next serial number in the sequence | ||
570 | */ | ||
571 | static dc_serial_t next_serialnum(void) | ||
572 | { | ||
573 | dc_serial_t serialnum = MAX(dircache.last_serialnum + 1, 1); | ||
574 | dircache.last_serialnum = serialnum; | ||
575 | return serialnum; | ||
576 | } | ||
577 | |||
578 | /** | ||
579 | * return the dircache volume pointer for the special index | ||
580 | */ | ||
581 | static struct dircache_volume * get_idx_dcvolp(int idx) | ||
582 | { | ||
583 | if (idx >= 0) | ||
584 | return NULL; | ||
585 | |||
586 | return &dircache.dcvol[IF_MV_VOL(-idx - 1)]; | ||
587 | } | ||
588 | |||
589 | /** | ||
590 | * return the cache entry referenced by idx (NULL if outside buffer) | ||
591 | */ | ||
592 | static struct dircache_entry * get_entry(int idx) | ||
593 | { | ||
594 | if (idx <= 0 || (unsigned int)idx > dircache.numentries) | ||
595 | return NULL; | ||
223 | 596 | ||
224 | return next_entry; | 597 | return &dircache_runinfo.pentry[idx]; |
225 | } | 598 | } |
226 | 599 | ||
227 | /** | 600 | /** |
228 | * Internal function to allocate a dircache_entry and set | 601 | * return the index of the cache entry (0 if outside buffer) |
229 | * ->next entry pointers. | ||
230 | */ | 602 | */ |
231 | static struct dircache_entry* dircache_gen_next(struct dircache_entry *ce) | 603 | static int get_index(const struct dircache_entry *ce) |
232 | { | 604 | { |
233 | struct dircache_entry *next_entry; | 605 | if (!PTR_IN_ARRAY(dircache_runinfo.pentry + 1, ce, |
606 | dircache.numentries + 1)) | ||
607 | { | ||
608 | return 0; | ||
609 | } | ||
610 | |||
611 | return ce - dircache_runinfo.pentry; | ||
612 | } | ||
234 | 613 | ||
235 | if ( (next_entry = allocate_entry()) == NULL) | 614 | /** |
615 | * return the sublist down pointer for the sublist that contains entry 'idx' | ||
616 | */ | ||
617 | static int * get_downidxp(int idx) | ||
618 | { | ||
619 | /* NOTE: 'idx' must refer to a directory or the result is undefined */ | ||
620 | if (idx == 0 || idx < -NUM_VOLUMES) | ||
236 | return NULL; | 621 | return NULL; |
237 | next_entry->up = ce->up; | 622 | |
238 | ce->next = next_entry; | 623 | if (idx > 0) |
239 | 624 | { | |
240 | return next_entry; | 625 | /* a normal entry */ |
626 | struct dircache_entry *ce = get_entry(idx); | ||
627 | return ce ? &ce->down : NULL; | ||
628 | } | ||
629 | else | ||
630 | { | ||
631 | /* a volume root */ | ||
632 | return &get_idx_dcvolp(idx)->root_down; | ||
633 | } | ||
241 | } | 634 | } |
242 | 635 | ||
243 | /* | 636 | /** |
244 | * Internal function to allocate a dircache_entry and set | 637 | * return a pointer to the index referencing the cache entry that 'idx' |
245 | * ->down entry pointers. | 638 | * references |
246 | */ | 639 | */ |
247 | static struct dircache_entry* dircache_gen_down(struct dircache_entry *ce) | 640 | static int * get_previdxp(int idx) |
248 | { | 641 | { |
249 | struct dircache_entry *next_entry; | 642 | struct dircache_entry *ce = get_entry(idx); |
250 | 643 | ||
251 | if ( (next_entry = allocate_entry()) == NULL) | 644 | int *prevp = get_downidxp(ce->up); |
645 | if (!prevp) | ||
252 | return NULL; | 646 | return NULL; |
253 | next_entry->up = ce; | 647 | |
254 | ce->down = next_entry; | 648 | while (1) |
255 | 649 | { | |
256 | return next_entry; | 650 | int next = *prevp; |
651 | if (!next || next == idx) | ||
652 | break; | ||
653 | |||
654 | prevp = &get_entry(next)->next; | ||
655 | } | ||
656 | |||
657 | return prevp; | ||
658 | } | ||
659 | |||
660 | /** | ||
661 | * if required, adjust the lists and directory read of any scan and build in | ||
662 | * progress | ||
663 | */ | ||
664 | static void sab_sync_scan(struct sab *sabp, int *prevp, int *nextp) | ||
665 | { | ||
666 | struct sab_component *abovep = NULL; | ||
667 | FOR_EACH_SAB_COMP(sabp, p) | ||
668 | { | ||
669 | if (nextp != p->prevp) | ||
670 | { | ||
671 | abovep = p; | ||
672 | continue; | ||
673 | } | ||
674 | |||
675 | /* removing an item being scanned; set the component position to the | ||
676 | entry before this */ | ||
677 | p->prevp = prevp; | ||
678 | |||
679 | if (p == sabp->top) | ||
680 | { | ||
681 | /* removed at item in the directory who's immediate contents are | ||
682 | being scanned */ | ||
683 | if (prevp == p->downp) | ||
684 | { | ||
685 | /* was first item; rewind it */ | ||
686 | dircache_rewinddir_internal(&sabp->info); | ||
687 | } | ||
688 | else | ||
689 | { | ||
690 | struct dircache_entry *ceprev = | ||
691 | container_of(prevp, struct dircache_entry, next); | ||
692 | #ifdef DIRCACHE_NATIVE | ||
693 | sabp->info.fatfile.e.entry = ceprev->direntry; | ||
694 | sabp->info.fatfile.e.entries = ceprev->direntries; | ||
695 | #endif | ||
696 | } | ||
697 | } | ||
698 | else if (abovep) | ||
699 | { | ||
700 | /* the directory being scanned or a parent of it has been removed; | ||
701 | abort its build or cache traversal */ | ||
702 | abovep->idx = 0; | ||
703 | } | ||
704 | |||
705 | break; | ||
706 | } | ||
707 | } | ||
708 | |||
709 | /** | ||
710 | * get a pointer to an allocated name given a cache index | ||
711 | */ | ||
712 | static inline unsigned char * get_name(int nameidx) | ||
713 | { | ||
714 | return &dircache_runinfo.pname[nameidx]; | ||
715 | } | ||
716 | |||
717 | /** | ||
718 | * get the cache buffer index of the given name | ||
719 | */ | ||
720 | static inline int get_nameidx(const unsigned char *pname) | ||
721 | { | ||
722 | return pname - dircache_runinfo.pname; | ||
723 | } | ||
724 | |||
725 | /** | ||
726 | * copy the entry's name to a buffer (which assumed to be of sufficient size) | ||
727 | */ | ||
728 | static void entry_name_copy(char *dst, const struct dircache_entry *ce) | ||
729 | { | ||
730 | if (LIKELY(!ce->tinyname)) | ||
731 | { | ||
732 | strmemcpy(dst, get_name(ce->name), ce->length); | ||
733 | return; | ||
734 | } | ||
735 | |||
736 | const unsigned char *src = ce->namebuf; | ||
737 | size_t len = 0; | ||
738 | while (len++ < MAX_TINYNAME && *src) | ||
739 | *dst++ = *src++; | ||
740 | |||
741 | *dst = '\0'; | ||
742 | } | ||
743 | |||
744 | /** | ||
745 | * set the namesfree hint to a new position | ||
746 | */ | ||
747 | static void set_namesfree_hint(const unsigned char *hintp) | ||
748 | { | ||
749 | int hintidx = get_nameidx(hintp); | ||
750 | |||
751 | if (hintidx >= (int)(dircache.names + dircache.sizenames)) | ||
752 | hintidx = dircache.names; | ||
753 | |||
754 | dircache.nextnamefree = hintidx; | ||
257 | } | 755 | } |
258 | 756 | ||
259 | /** | 757 | /** |
260 | * Returns true if there is an event waiting in the queue | 758 | * allocate a buffer to use for a new name |
261 | * that requires the current operation to be aborted. | ||
262 | */ | 759 | */ |
263 | static bool check_event_queue(void) | 760 | static int alloc_name(size_t size) |
761 | { | ||
762 | int nameidx = 0; | ||
763 | |||
764 | if (dircache.namesfree >= size) | ||
765 | { | ||
766 | /* scan for a free gap starting at the hint point - first fit */ | ||
767 | unsigned char *start = get_name(dircache.nextnamefree), *p = start; | ||
768 | unsigned char *namesend = get_name(dircache.names + dircache.sizenames); | ||
769 | size_t gapsize = 0; | ||
770 | |||
771 | while (gapsize < size) | ||
772 | { | ||
773 | if ((p = memchr(p, 0xff, namesend - p))) | ||
774 | { | ||
775 | /* found a sentinel; see if there are enough in a row */ | ||
776 | gapsize = 1; | ||
777 | while (*++p == 0xff && gapsize < size) | ||
778 | gapsize++; | ||
779 | } | ||
780 | else | ||
781 | { | ||
782 | if (namesend == start) | ||
783 | break; /* exhausted */ | ||
784 | |||
785 | /* wrap */ | ||
786 | namesend = start; | ||
787 | p = get_name(dircache.names); | ||
788 | |||
789 | if (p == namesend) | ||
790 | break; /* initial hint was at names start */ | ||
791 | } | ||
792 | } | ||
793 | |||
794 | if (gapsize >= size) | ||
795 | { | ||
796 | unsigned char *namep = p - gapsize; | ||
797 | nameidx = get_nameidx(namep); | ||
798 | |||
799 | if (*p == 0xff) | ||
800 | { | ||
801 | /* if only a tiny block remains after buffer, claim it too */ | ||
802 | size_t tinysize = 1; | ||
803 | while (*++p == 0xff && tinysize <= MAX_TINYNAME) | ||
804 | tinysize++; | ||
805 | |||
806 | if (tinysize <= MAX_TINYNAME) | ||
807 | { | ||
808 | /* mark with tiny block sentinel */ | ||
809 | memset(p - tinysize, 0xfe, tinysize); | ||
810 | size += tinysize; | ||
811 | } | ||
812 | } | ||
813 | |||
814 | dircache.namesfree -= size; | ||
815 | dircache.sizeused += size; | ||
816 | set_namesfree_hint(namep + size); | ||
817 | } | ||
818 | } | ||
819 | |||
820 | if (!nameidx) | ||
821 | { | ||
822 | /* no sufficiently long free gaps; allocate anew */ | ||
823 | if (dircache_buf_remaining() <= size) | ||
824 | { | ||
825 | dircache.last_size = 0; | ||
826 | return 0; | ||
827 | } | ||
828 | |||
829 | dircache.names -= size; | ||
830 | dircache.sizenames += size; | ||
831 | nameidx = dircache.names; | ||
832 | dircache.size += size; | ||
833 | dircache.sizeused += size; | ||
834 | *get_name(dircache.names - 1) = 0; | ||
835 | } | ||
836 | |||
837 | return nameidx; | ||
838 | } | ||
839 | |||
840 | /** | ||
841 | * mark a name as free and note that its bytes are available | ||
842 | */ | ||
843 | static void free_name(int nameidx, size_t size) | ||
844 | { | ||
845 | unsigned char *beg = get_name(nameidx); | ||
846 | unsigned char *end = beg + size; | ||
847 | |||
848 | /* merge with any adjacent tiny blocks */ | ||
849 | while (beg[-1] == 0xfe) | ||
850 | --beg; | ||
851 | |||
852 | while (end[1] == 0xfe) | ||
853 | ++end; | ||
854 | |||
855 | size = end - beg; | ||
856 | memset(beg, 0xff, size); | ||
857 | dircache.namesfree += size; | ||
858 | dircache.sizeused -= size; | ||
859 | set_namesfree_hint(beg); | ||
860 | } | ||
861 | |||
862 | /** | ||
863 | * allocate and assign a name to the entry | ||
864 | */ | ||
865 | static bool entry_assign_name(struct dircache_entry *ce, | ||
866 | const unsigned char *name, size_t size) | ||
867 | { | ||
868 | unsigned char *copyto; | ||
869 | |||
870 | if (size <= MAX_TINYNAME) | ||
871 | { | ||
872 | copyto = ce->namebuf; | ||
873 | |||
874 | if (size < MAX_TINYNAME) | ||
875 | copyto[size] = '\0'; | ||
876 | |||
877 | ce->tinyname = 1; | ||
878 | } | ||
879 | else | ||
880 | { | ||
881 | if (size > DC_MAX_NAME) | ||
882 | size = DC_MAX_NAME; | ||
883 | |||
884 | int nameidx = alloc_name(size); | ||
885 | if (!nameidx) | ||
886 | return false; | ||
887 | |||
888 | copyto = get_name(nameidx); | ||
889 | |||
890 | ce->tinyname = 0; | ||
891 | ce->name = nameidx; | ||
892 | ce->length = size; | ||
893 | } | ||
894 | |||
895 | memcpy(copyto, name, size); | ||
896 | return true; | ||
897 | } | ||
898 | |||
899 | /** | ||
900 | * free the name for the entry | ||
901 | */ | ||
902 | static void entry_unassign_name(struct dircache_entry *ce) | ||
903 | { | ||
904 | if (!ce->tinyname) | ||
905 | free_name(ce->name, ce->length); | ||
906 | } | ||
907 | |||
908 | /** | ||
909 | * assign a new name to the entry | ||
910 | */ | ||
911 | static bool entry_reassign_name(struct dircache_entry *ce, | ||
912 | const unsigned char *newname) | ||
913 | { | ||
914 | size_t oldlen = ce->tinyname ? 0 : ce->length; | ||
915 | size_t newlen = strlen(newname); | ||
916 | |||
917 | if (oldlen == newlen || (oldlen == 0 && newlen <= MAX_TINYNAME)) | ||
918 | { | ||
919 | char *p = mempcpy(oldlen == 0 ? ce->namebuf : get_name(ce->name), | ||
920 | newname, newlen); | ||
921 | if (newlen < MAX_TINYNAME) | ||
922 | *p = '\0'; | ||
923 | return true; | ||
924 | } | ||
925 | |||
926 | /* needs a new name allocation; if the new name fits in the freed block, | ||
927 | it will use it immediately without a lengthy search */ | ||
928 | entry_unassign_name(ce); | ||
929 | return entry_assign_name(ce, newname, newlen); | ||
930 | } | ||
931 | |||
932 | /** | ||
933 | * allocate a dircache_entry from memory using freed ones if available | ||
934 | */ | ||
935 | static int alloc_entry(struct dircache_entry **res) | ||
936 | { | ||
937 | struct dircache_entry *ce; | ||
938 | int idx = dircache.free_list; | ||
939 | |||
940 | if (idx) | ||
941 | { | ||
942 | /* reuse a freed entry */ | ||
943 | ce = get_entry(idx); | ||
944 | dircache.free_list = ce->next; | ||
945 | } | ||
946 | else if (dircache_buf_remaining() > ENTRYSIZE) | ||
947 | { | ||
948 | /* allocate a new one */ | ||
949 | idx = ++dircache.numentries; | ||
950 | dircache.size += ENTRYSIZE; | ||
951 | ce = get_entry(idx); | ||
952 | } | ||
953 | else | ||
954 | { | ||
955 | dircache.last_size = 0; | ||
956 | *res = NULL; | ||
957 | return 0; | ||
958 | } | ||
959 | |||
960 | dircache.sizeused += ENTRYSIZE; | ||
961 | |||
962 | ce->next = 0; | ||
963 | ce->up = 0; | ||
964 | ce->down = 0; | ||
965 | ce->length = 0; | ||
966 | ce->frontier = FRONTIER_SETTLED; | ||
967 | ce->serialnum = next_serialnum(); | ||
968 | |||
969 | *res = ce; | ||
970 | return idx; | ||
971 | } | ||
972 | |||
973 | /** | ||
974 | * free an entry's allocations in the cache; must not be linked to anything | ||
975 | * by this time (orphan!) | ||
976 | */ | ||
977 | static void free_orphan_entry(struct dircache_runinfo_volume *dcrivolp, | ||
978 | struct dircache_entry *ce, int idx) | ||
979 | { | ||
980 | if (dcrivolp) | ||
981 | { | ||
982 | /* was an established entry; find any associated resolved binding and | ||
983 | dissolve it; bindings are kept strictly synchronized with changes | ||
984 | to the storage so a simple serial number comparison is sufficient */ | ||
985 | struct file_base_binding *prevp = NULL; | ||
986 | FOR_EACH_BINDING(dcrivolp->resolved0, p) | ||
987 | { | ||
988 | if (p == dcrivolp->queued0) | ||
989 | break; | ||
990 | |||
991 | if (ce->serialnum == p->info.dcfile.serialnum) | ||
992 | { | ||
993 | binding_dissolve(prevp, p); | ||
994 | break; | ||
995 | } | ||
996 | |||
997 | prevp = p; | ||
998 | } | ||
999 | } | ||
1000 | |||
1001 | entry_unassign_name(ce); | ||
1002 | |||
1003 | /* no serialnum says "it's free" (for cache-wide iterators) */ | ||
1004 | ce->serialnum = 0; | ||
1005 | |||
1006 | /* add to free list */ | ||
1007 | ce->next = dircache.free_list; | ||
1008 | dircache.free_list = idx; | ||
1009 | dircache.sizeused -= ENTRYSIZE; | ||
1010 | } | ||
1011 | |||
1012 | /** | ||
1013 | * allocates a new entry of with the name specified by 'basename' | ||
1014 | */ | ||
1015 | static int create_entry(const char *basename, | ||
1016 | struct dircache_entry **res) | ||
1017 | { | ||
1018 | int idx = alloc_entry(res); | ||
1019 | if (!idx) | ||
1020 | { | ||
1021 | logf("size limit reached (entry)"); | ||
1022 | return 0; /* full */ | ||
1023 | } | ||
1024 | |||
1025 | if (!entry_assign_name(*res, basename, strlen(basename))) | ||
1026 | { | ||
1027 | free_orphan_entry(NULL, *res, idx); | ||
1028 | logf("size limit reached (name)"); | ||
1029 | return 0; /* really full! */ | ||
1030 | } | ||
1031 | |||
1032 | return idx; | ||
1033 | } | ||
1034 | |||
1035 | /** | ||
1036 | * unlink the entry at *prevp and adjust the scanner if needed | ||
1037 | */ | ||
1038 | static void remove_entry(struct dircache_runinfo_volume *dcrivolp, | ||
1039 | struct dircache_entry *ce, int *prevp) | ||
1040 | { | ||
1041 | /* unlink it from its list */ | ||
1042 | *prevp = ce->next; | ||
1043 | |||
1044 | if (dcrivolp) | ||
1045 | { | ||
1046 | /* adjust scanner iterator if needed */ | ||
1047 | struct sab *sabp = dcrivolp->sabp; | ||
1048 | if (sabp) | ||
1049 | sab_sync_scan(sabp, prevp, &ce->next); | ||
1050 | } | ||
1051 | } | ||
1052 | |||
1053 | /** | ||
1054 | * free the entire subtree in the referenced parent down index | ||
1055 | */ | ||
1056 | static void free_subentries(struct dircache_runinfo_volume *dcrivolp, int *downp) | ||
1057 | { | ||
1058 | while (1) | ||
1059 | { | ||
1060 | int idx = *downp; | ||
1061 | struct dircache_entry *ce = get_entry(idx); | ||
1062 | if (!ce) | ||
1063 | break; | ||
1064 | |||
1065 | if ((ce->attr & ATTR_DIRECTORY) && ce->down) | ||
1066 | free_subentries(dcrivolp, &ce->down); | ||
1067 | |||
1068 | remove_entry(dcrivolp, ce, downp); | ||
1069 | free_orphan_entry(dcrivolp, ce, idx); | ||
1070 | } | ||
1071 | } | ||
1072 | |||
1073 | /** | ||
1074 | * free the specified file entry and its children | ||
1075 | */ | ||
1076 | static void free_file_entry(struct file_base_info *infop) | ||
1077 | { | ||
1078 | int idx = infop->dcfile.idx; | ||
1079 | if (idx <= 0) | ||
1080 | return; /* can't remove a root/invalid */ | ||
1081 | |||
1082 | struct dircache_runinfo_volume *dcrivolp = DCRIVOL(infop); | ||
1083 | |||
1084 | struct dircache_entry *ce = get_entry(idx); | ||
1085 | if ((ce->attr & ATTR_DIRECTORY) && ce->down) | ||
1086 | { | ||
1087 | /* gonna get all this contents (normally the "." and "..") */ | ||
1088 | free_subentries(dcrivolp, &ce->down); | ||
1089 | } | ||
1090 | |||
1091 | remove_entry(dcrivolp, ce, get_previdxp(idx)); | ||
1092 | free_orphan_entry(dcrivolp, ce, idx); | ||
1093 | } | ||
1094 | |||
1095 | /** | ||
1096 | * insert the new entry into the parent, sorted into position | ||
1097 | */ | ||
1098 | static void insert_file_entry(struct file_base_info *dirinfop, | ||
1099 | struct dircache_entry *ce) | ||
1100 | { | ||
1101 | /* DIRCACHE_NATIVE: the entires are sorted into the spot it would be on | ||
1102 | * the storage medium based upon the directory entry number, in-progress | ||
1103 | * scans will catch it or miss it in just the same way they would if | ||
1104 | * directly scanning the disk. If this is behind an init scan, it gets | ||
1105 | * added anyway; if in front of it, then scanning will compare what it | ||
1106 | * finds in order to avoid adding a duplicate. | ||
1107 | * | ||
1108 | * All others: the order of entries of the host filesystem is not known so | ||
1109 | * this must be placed at the end so that a build scan won't miss it and | ||
1110 | * add a duplicate since it will be comparing any entries it finds in front | ||
1111 | * of it. | ||
1112 | */ | ||
1113 | int diridx = dirinfop->dcfile.idx; | ||
1114 | int *nextp = get_downidxp(diridx); | ||
1115 | |||
1116 | while (8675309) | ||
1117 | { | ||
1118 | struct dircache_entry *nextce = get_entry(*nextp); | ||
1119 | if (!nextce) | ||
1120 | break; | ||
1121 | |||
1122 | #ifdef DIRCACHE_NATIVE | ||
1123 | if (nextce->direntry > ce->direntry) | ||
1124 | break; | ||
1125 | |||
1126 | /* now, nothing should be equal to ours or that is a bug since it | ||
1127 | would already exist (and it shouldn't because it's just been | ||
1128 | created or moved) */ | ||
1129 | #endif /* DIRCACHE_NATIVE */ | ||
1130 | |||
1131 | nextp = &nextce->next; | ||
1132 | } | ||
1133 | |||
1134 | ce->up = diridx; | ||
1135 | ce->next = *nextp; | ||
1136 | *nextp = get_index(ce); | ||
1137 | } | ||
1138 | |||
1139 | /** | ||
1140 | * unlink the entry from its parent and return its pointer to the caller | ||
1141 | */ | ||
1142 | static struct dircache_entry * remove_file_entry(struct file_base_info *infop) | ||
1143 | { | ||
1144 | struct dircache_runinfo_volume *dcrivolp = DCRIVOL(infop); | ||
1145 | struct dircache_entry *ce = get_entry(infop->dcfile.idx); | ||
1146 | remove_entry(dcrivolp, ce, get_previdxp(infop->dcfile.idx)); | ||
1147 | return ce; | ||
1148 | } | ||
1149 | |||
1150 | /** | ||
1151 | * set the frontier indicator for the given cache index | ||
1152 | */ | ||
1153 | static void establish_frontier(int idx, uint32_t code) | ||
1154 | { | ||
1155 | if (idx < 0) | ||
1156 | { | ||
1157 | int volume = IF_MV_VOL(-idx - 1); | ||
1158 | uint32_t val = dircache.dcvol[volume].frontier; | ||
1159 | if (code & FRONTIER_RENEW) | ||
1160 | val &= ~FRONTIER_ZONED; | ||
1161 | dircache.dcvol[volume].frontier = code | (val & FRONTIER_ZONED); | ||
1162 | } | ||
1163 | else if (idx > 0) | ||
1164 | { | ||
1165 | struct dircache_entry *ce = get_entry(idx); | ||
1166 | uint32_t val = ce->frontier; | ||
1167 | if (code & FRONTIER_RENEW) | ||
1168 | val &= ~FRONTIER_ZONED; | ||
1169 | ce->frontier = code | (val & FRONTIER_ZONED); | ||
1170 | } | ||
1171 | } | ||
1172 | |||
1173 | /** | ||
1174 | * remove all messages from the queue, responding to anyone waiting | ||
1175 | */ | ||
1176 | static void clear_dircache_queue(void) | ||
264 | { | 1177 | { |
265 | struct queue_event ev; | 1178 | struct queue_event ev; |
266 | 1179 | ||
267 | if(!queue_peek(&dircache_queue, &ev)) | 1180 | while (1) |
268 | return false; | 1181 | { |
269 | 1182 | queue_wait_w_tmo(&dircache_queue, &ev, 0); | |
270 | switch (ev.id) | 1183 | if (ev.id == SYS_TIMEOUT) |
271 | { | 1184 | break; |
272 | case DIRCACHE_STOP: | 1185 | |
273 | case SYS_USB_CONNECTED: | 1186 | /* respond to any synchronous build queries; since we're already |
274 | #ifdef HAVE_HOTSWAP | 1187 | building and thusly allocated, any additional requests can be |
275 | case SYS_FS_CHANGED: | 1188 | processed async */ |
276 | #endif | 1189 | if (ev.id == DCM_BUILD) |
277 | return true; | 1190 | { |
1191 | int *rcp = (int *)ev.data; | ||
1192 | if (rcp) | ||
1193 | *rcp = 0; | ||
1194 | } | ||
278 | } | 1195 | } |
279 | |||
280 | return false; | ||
281 | } | 1196 | } |
282 | 1197 | ||
283 | #if (CONFIG_PLATFORM & PLATFORM_NATIVE) | 1198 | /** |
284 | /* scan and build static data (avoid redundancy on stack) */ | 1199 | * service dircache_queue during a scan and build |
285 | static struct | 1200 | */ |
1201 | static void process_events(void) | ||
286 | { | 1202 | { |
287 | #ifdef HAVE_MULTIVOLUME | 1203 | yield(); |
288 | int volume; | 1204 | |
289 | #endif | 1205 | /* only count externally generated commands */ |
290 | struct fat_dir *dir; | 1206 | if (!queue_peek_ex(&dircache_queue, NULL, 0, QPEEK_FILTER1(DCM_BUILD))) |
291 | struct fat_direntry *direntry; | 1207 | return; |
292 | }sab; | 1208 | |
293 | 1209 | clear_dircache_queue(); | |
294 | static int sab_process_dir(unsigned long startcluster, struct dircache_entry *ce) | 1210 | |
295 | { | 1211 | /* this reminds us to keep moving after we're done here; a volume we passed |
296 | /* normally, opendir expects a full fat_dir as parent but in our case, | 1212 | up earlier could have been mounted and need refreshing; it just condenses |
297 | * it's completely useless because we don't modify anything | 1213 | a slew of requests into one and isn't mistaken for an externally generated |
298 | * WARNING: this heavily relies on current FAT implementation ! */ | 1214 | command */ |
299 | 1215 | queue_post(&dircache_queue, DCM_PROCEED, 0); | |
300 | /* those field are necessary to update the FAT entry in case of modification | 1216 | } |
301 | here we don't touch anything so we put dummy values */ | 1217 | |
302 | sab.dir->entry = 0; | 1218 | #if defined (DIRCACHE_NATIVE) |
303 | sab.dir->entrycount = 0; | 1219 | /** |
304 | sab.dir->file.firstcluster = 0; | 1220 | * scan and build the contents of a subdirectory |
305 | /* open directory */ | 1221 | */ |
306 | int rc = fat_opendir(IF_MV(sab.volume,) sab.dir, startcluster, sab.dir); | 1222 | static void sab_process_sub(struct sab *sabp) |
307 | if(rc < 0) | 1223 | { |
308 | { | 1224 | struct fat_direntry *const fatentp = get_dir_fatent(); |
309 | logf("fat_opendir failed: %d", rc); | 1225 | struct filestr_base *const streamp = &sabp->stream; |
310 | return rc; | 1226 | struct file_base_info *const infop = &sabp->info; |
1227 | |||
1228 | int idx = infop->dcfile.idx; | ||
1229 | int *downp = get_downidxp(idx); | ||
1230 | if (!downp) | ||
1231 | return; | ||
1232 | |||
1233 | while (1) | ||
1234 | { | ||
1235 | struct sab_component *compp = --sabp->top; | ||
1236 | compp->idx = idx; | ||
1237 | compp->downp = downp; | ||
1238 | compp->prevp = downp; | ||
1239 | |||
1240 | /* open directory stream */ | ||
1241 | filestr_base_init(streamp); | ||
1242 | fileobj_fileop_open(streamp, infop, FO_DIRECTORY); | ||
1243 | fat_rewind(&streamp->fatstr); | ||
1244 | uncached_rewinddir_internal(infop); | ||
1245 | |||
1246 | const long dircluster = streamp->infop->fatfile.firstcluster; | ||
1247 | |||
1248 | /* first pass: read directory */ | ||
1249 | while (1) | ||
1250 | { | ||
1251 | if (sabp->stack + 1 < sabp->stackend) | ||
1252 | { | ||
1253 | /* release control and process queued events */ | ||
1254 | dircache_unlock(); | ||
1255 | process_events(); | ||
1256 | dircache_lock(); | ||
1257 | |||
1258 | if (sabp->quit || !compp->idx) | ||
1259 | break; | ||
1260 | } | ||
1261 | /* else an immediate-contents directory scan */ | ||
1262 | |||
1263 | int rc = uncached_readdir_internal(streamp, infop, fatentp); | ||
1264 | if (rc <= 0) | ||
1265 | { | ||
1266 | if (rc < 0) | ||
1267 | sabp->quit = true; | ||
1268 | else | ||
1269 | compp->prevp = downp; /* rewind list */ | ||
1270 | |||
1271 | break; | ||
1272 | } | ||
1273 | |||
1274 | struct dircache_entry *ce; | ||
1275 | int prev = *compp->prevp; | ||
1276 | |||
1277 | if (prev) | ||
1278 | { | ||
1279 | /* there are entries ahead of us; they will be what was just | ||
1280 | read or something to be subsequently read; if it belongs | ||
1281 | ahead of this one, insert a new entry before it; if it's | ||
1282 | the entry just scanned, do nothing further and continue | ||
1283 | with the next */ | ||
1284 | ce = get_entry(prev); | ||
1285 | if (ce->direntry == infop->fatfile.e.entry) | ||
1286 | { | ||
1287 | compp->prevp = &ce->next; | ||
1288 | continue; /* already there */ | ||
1289 | } | ||
1290 | } | ||
1291 | |||
1292 | int idx = create_entry(fatentp->name, &ce); | ||
1293 | if (!idx) | ||
1294 | { | ||
1295 | sabp->quit = true; | ||
1296 | break; | ||
1297 | } | ||
1298 | |||
1299 | /* link it in */ | ||
1300 | ce->up = compp->idx; | ||
1301 | ce->next = prev; | ||
1302 | *compp->prevp = idx; | ||
1303 | compp->prevp = &ce->next; | ||
1304 | |||
1305 | if (!(fatentp->attr & ATTR_DIRECTORY)) | ||
1306 | ce->filesize = fatentp->filesize; | ||
1307 | else if (!is_dotdir_name(fatentp->name)) | ||
1308 | ce->frontier = FRONTIER_NEW; /* this needs scanning */ | ||
1309 | |||
1310 | /* copy remaining FS info */ | ||
1311 | ce->direntry = infop->fatfile.e.entry; | ||
1312 | ce->direntries = infop->fatfile.e.entries; | ||
1313 | ce->attr = fatentp->attr; | ||
1314 | ce->firstcluster = fatentp->firstcluster; | ||
1315 | ce->wrtdate = fatentp->wrtdate; | ||
1316 | ce->wrttime = fatentp->wrttime; | ||
1317 | |||
1318 | /* resolve queued user bindings */ | ||
1319 | infop->fatfile.firstcluster = fatentp->firstcluster; | ||
1320 | infop->fatfile.dircluster = dircluster; | ||
1321 | infop->dcfile.idx = idx; | ||
1322 | infop->dcfile.serialnum = ce->serialnum; | ||
1323 | binding_resolve(infop); | ||
1324 | } /* end while */ | ||
1325 | |||
1326 | close_stream_internal(streamp); | ||
1327 | |||
1328 | if (sabp->quit) | ||
1329 | return; | ||
1330 | |||
1331 | establish_frontier(compp->idx, FRONTIER_SETTLED); | ||
1332 | |||
1333 | /* second pass: "recurse!" */ | ||
1334 | struct dircache_entry *ce = NULL; | ||
1335 | |||
1336 | while (1) | ||
1337 | { | ||
1338 | idx = compp->idx && compp > sabp->stack ? *compp->prevp : 0; | ||
1339 | if (idx) | ||
1340 | { | ||
1341 | ce = get_entry(idx); | ||
1342 | compp->prevp = &ce->next; | ||
1343 | |||
1344 | if (ce->frontier != FRONTIER_SETTLED) | ||
1345 | break; | ||
1346 | } | ||
1347 | else | ||
1348 | { | ||
1349 | /* directory completed or removed/deepest level */ | ||
1350 | compp = ++sabp->top; | ||
1351 | if (compp >= sabp->stackend) | ||
1352 | return; /* scan completed/initial directory removed */ | ||
1353 | } | ||
1354 | } | ||
1355 | |||
1356 | /* even if it got zoned from outside it is about to be scanned in | ||
1357 | its entirety and may be considered new again */ | ||
1358 | ce->frontier = FRONTIER_NEW; | ||
1359 | downp = &ce->down; | ||
1360 | |||
1361 | /* set up info for next open | ||
1362 | * IF_MV: "volume" was set when scan began */ | ||
1363 | infop->fatfile.firstcluster = ce->firstcluster; | ||
1364 | infop->fatfile.dircluster = dircluster; | ||
1365 | infop->fatfile.e.entry = ce->direntry; | ||
1366 | infop->fatfile.e.entries = ce->direntries; | ||
1367 | } /* end while */ | ||
1368 | } | ||
1369 | |||
1370 | /** | ||
1371 | * scan and build the contents of a directory or volume root | ||
1372 | */ | ||
1373 | static void sab_process_dir(struct file_base_info *infop, bool issab) | ||
1374 | { | ||
1375 | /* infop should have been fully opened meaning that all its parent | ||
1376 | directory information is filled in and intact; the binding information | ||
1377 | should also filled in beforehand */ | ||
1378 | |||
1379 | /* allocate the stack right now to the max demand */ | ||
1380 | struct dirsab | ||
1381 | { | ||
1382 | struct sab sab; | ||
1383 | struct sab_component stack[issab ? DIRCACHE_MAX_DEPTH : 1]; | ||
1384 | } dirsab; | ||
1385 | struct sab *sabp = &dirsab.sab; | ||
1386 | |||
1387 | sabp->quit = false; | ||
1388 | sabp->stackend = &sabp->stack[ARRAYLEN(dirsab.stack)]; | ||
1389 | sabp->top = sabp->stackend; | ||
1390 | sabp->info = *infop; | ||
1391 | |||
1392 | if (issab) | ||
1393 | DCRIVOL(infop)->sabp = sabp; | ||
1394 | |||
1395 | establish_frontier(infop->dcfile.idx, FRONTIER_NEW | FRONTIER_RENEW); | ||
1396 | sab_process_sub(sabp); | ||
1397 | |||
1398 | if (issab) | ||
1399 | DCRIVOL(infop)->sabp = NULL; | ||
1400 | } | ||
1401 | |||
1402 | /** | ||
1403 | * scan and build the entire tree for a volume | ||
1404 | */ | ||
1405 | static void sab_process_volume(struct dircache_volume *dcvolp) | ||
1406 | { | ||
1407 | int rc; | ||
1408 | |||
1409 | int volume = IF_MV_VOL(dcvolp - dircache.dcvol); | ||
1410 | int idx = -volume - 1; | ||
1411 | |||
1412 | logf("dircache - building volume %d", volume); | ||
1413 | |||
1414 | /* gather everything sab_process_dir() needs in order to begin a scan */ | ||
1415 | struct file_base_info info; | ||
1416 | rc = fat_open_rootdir(IF_MV(volume,) &info.fatfile); | ||
1417 | if (rc < 0) | ||
1418 | { | ||
1419 | /* probably not mounted */ | ||
1420 | logf("SAB - no root %d: %d", volume, rc); | ||
1421 | establish_frontier(idx, FRONTIER_NEW); | ||
1422 | return; | ||
311 | } | 1423 | } |
312 | 1424 | ||
313 | /* first pass : read dir */ | 1425 | info.dcfile.idx = idx; |
314 | struct dircache_entry *first_ce = ce; | 1426 | info.dcfile.serialnum = dcvolp->serialnum; |
315 | 1427 | sab_process_dir(&info, true); | |
316 | /* read through directory */ | 1428 | } |
317 | while((rc = fat_getnext(sab.dir, sab.direntry)) >= 0 && sab.direntry->name[0]) | 1429 | |
1430 | /** | ||
1431 | * this function is the back end to the public API's like readdir() | ||
1432 | */ | ||
1433 | int dircache_readdir_dirent(struct filestr_base *stream, | ||
1434 | struct dirscan_info *scanp, | ||
1435 | struct dirent *entry) | ||
1436 | { | ||
1437 | struct file_base_info *dirinfop = stream->infop; | ||
1438 | |||
1439 | if (!dirinfop->dcfile.serialnum) | ||
1440 | goto read_uncached; /* no parent cached => no entries cached */ | ||
1441 | |||
1442 | struct dircache_volume *dcvolp = DCVOL(dirinfop); | ||
1443 | |||
1444 | int diridx = dirinfop->dcfile.idx; | ||
1445 | unsigned int frontier = diridx <= 0 ? dcvolp->frontier : | ||
1446 | get_entry(diridx)->frontier; | ||
1447 | |||
1448 | /* if not settled, just readthrough; no binding information is needed for | ||
1449 | this; if it becomes settled, we'll get scan->dcfile caught up and do | ||
1450 | subsequent reads with the cache */ | ||
1451 | if (frontier != FRONTIER_SETTLED) | ||
1452 | goto read_uncached; | ||
1453 | |||
1454 | int idx = scanp->dcscan.idx; | ||
1455 | struct dircache_entry *ce; | ||
1456 | |||
1457 | unsigned int direntry = scanp->fatscan.entry; | ||
1458 | while (1) | ||
318 | { | 1459 | { |
319 | if(!strcmp(".", sab.direntry->name) || | 1460 | if (idx == 0 || direntry == FAT_RW_VAL) /* rewound? */ |
320 | !strcmp("..", sab.direntry->name)) | 1461 | { |
321 | continue; | 1462 | idx = diridx <= 0 ? dcvolp->root_down : get_entry(diridx)->down; |
1463 | break; | ||
1464 | } | ||
322 | 1465 | ||
323 | size_t size = strlen(sab.direntry->name) + 1; | 1466 | /* validate last entry scanned; it might have been replaced between |
324 | ce->d_name = (d_names_start -= size); | 1467 | calls or not there at all any more; if so, start the cache reader |
325 | ce->startcluster = sab.direntry->firstcluster; | 1468 | at the beginning and fast-forward to the correct point as indicated |
326 | ce->info.size = sab.direntry->filesize; | 1469 | by the FS scanner */ |
327 | ce->info.attribute = sab.direntry->attr; | 1470 | ce = get_entry(idx); |
328 | ce->info.wrtdate = sab.direntry->wrtdate; | 1471 | if (ce && ce->serialnum == scanp->dcscan.serialnum) |
329 | ce->info.wrttime = sab.direntry->wrttime; | 1472 | { |
1473 | idx = ce->next; | ||
1474 | break; | ||
1475 | } | ||
330 | 1476 | ||
331 | strcpy(ce->d_name, sab.direntry->name); | 1477 | idx = 0; |
332 | dircache_size += size; | 1478 | } |
333 | 1479 | ||
334 | if(ce->info.attribute & FAT_ATTR_DIRECTORY) | 1480 | while (1) |
335 | dircache_gen_down(ce); | 1481 | { |
336 | 1482 | ce = get_entry(idx); | |
337 | ce = dircache_gen_next(ce); | 1483 | if (!ce) |
338 | if(ce == NULL) | ||
339 | return -5; | ||
340 | |||
341 | /* When simulator is used, it's only safe to yield here. */ | ||
342 | if(thread_enabled) | ||
343 | { | 1484 | { |
344 | /* Stop if we got an external signal. */ | 1485 | empty_dirent(entry); |
345 | if(check_event_queue()) | 1486 | scanp->fatscan.entries = 0; |
346 | return -6; | 1487 | return 0; /* end of dir */ |
347 | yield(); | ||
348 | } | 1488 | } |
1489 | |||
1490 | if (ce->direntry > direntry || direntry == FAT_RW_VAL) | ||
1491 | break; /* cache reader is caught up to FS scan */ | ||
1492 | |||
1493 | idx = ce->next; | ||
349 | } | 1494 | } |
350 | 1495 | ||
351 | /* add "." and ".." */ | 1496 | /* basic dirent information */ |
352 | ce->d_name = dot; | 1497 | entry_name_copy(entry->d_name, ce); |
353 | ce->info.attribute = FAT_ATTR_DIRECTORY; | 1498 | entry->info.attr = ce->attr; |
354 | ce->startcluster = startcluster; | 1499 | entry->info.size = (ce->attr & ATTR_DIRECTORY) ? 0 : ce->filesize; |
355 | ce->info.size = 0; | 1500 | entry->info.wrtdate = ce->wrtdate; |
356 | ce->down = first_ce; | 1501 | entry->info.wrttime = ce->wrttime; |
357 | 1502 | ||
358 | ce = dircache_gen_next(ce); | 1503 | /* FS scan information */ |
359 | 1504 | scanp->fatscan.entry = ce->direntry; | |
360 | ce->d_name = dotdot; | 1505 | scanp->fatscan.entries = ce->direntries; |
361 | ce->info.attribute = FAT_ATTR_DIRECTORY; | 1506 | |
362 | ce->startcluster = (first_ce->up ? first_ce->up->startcluster : 0); | 1507 | /* dircache scan information */ |
363 | ce->info.size = 0; | 1508 | scanp->dcscan.idx = idx; |
364 | ce->down = first_ce->up; | 1509 | scanp->dcscan.serialnum = ce->serialnum; |
365 | 1510 | ||
366 | /* second pass: recurse ! */ | 1511 | /* return whether this needs decoding */ |
367 | ce = first_ce; | 1512 | int rc = ce->direntries == 1 ? 2 : 1; |
368 | 1513 | ||
369 | while(rc >= 0 && ce) | 1514 | yield(); |
370 | { | ||
371 | if(ce->d_name != NULL && ce->down != NULL && strcmp(ce->d_name, ".") | ||
372 | && strcmp(ce->d_name, "..")) | ||
373 | rc = sab_process_dir(ce->startcluster, ce->down); | ||
374 | |||
375 | ce = ce->next; | ||
376 | } | ||
377 | |||
378 | return rc; | 1515 | return rc; |
1516 | |||
1517 | read_uncached: | ||
1518 | dircache_dcfile_init(&scanp->dcscan); | ||
1519 | return uncached_readdir_dirent(stream, scanp, entry); | ||
379 | } | 1520 | } |
380 | 1521 | ||
381 | /* used during the generation */ | 1522 | /** |
382 | static struct fat_dir sab_fat_dir; | 1523 | * rewind the directory scan cursor |
1524 | */ | ||
1525 | void dircache_rewinddir_dirent(struct dirscan_info *scanp) | ||
1526 | { | ||
1527 | uncached_rewinddir_dirent(scanp); | ||
1528 | dircache_dcfile_init(&scanp->dcscan); | ||
1529 | } | ||
383 | 1530 | ||
384 | static int dircache_scan_and_build(IF_MV(int volume,) struct dircache_entry *ce) | 1531 | /** |
1532 | * this function is the back end to file API internal scanning, which requires | ||
1533 | * much more detail about the directory entries; this is allowed to make | ||
1534 | * assumptions about cache state because the cache will not be altered during | ||
1535 | * the scan process; an additional important property of internal scanning is | ||
1536 | * that any available binding information is not ignored even when a scan | ||
1537 | * directory is frontier zoned. | ||
1538 | */ | ||
1539 | int dircache_readdir_internal(struct filestr_base *stream, | ||
1540 | struct file_base_info *infop, | ||
1541 | struct fat_direntry *fatent) | ||
385 | { | 1542 | { |
386 | memset(ce, 0, sizeof(struct dircache_entry)); | 1543 | /* call with writer exclusion */ |
1544 | struct file_base_info *dirinfop = stream->infop; | ||
1545 | struct dircache_volume *dcvolp = DCVOL(dirinfop); | ||
387 | 1546 | ||
388 | #ifdef HAVE_MULTIVOLUME | 1547 | /* assume binding "not found" */ |
389 | if (volume > 0) | 1548 | infop->dcfile.serialnum = 0; |
390 | { | 1549 | |
391 | /* broken for 100+ volumes because the format string is too small | 1550 | /* is parent cached? if not, readthrough because nothing is here yet */ |
392 | * and we use that for size calculation */ | 1551 | if (!dirinfop->dcfile.serialnum) |
393 | const size_t max_len = VOL_ENUM_POS + 3; | 1552 | return uncached_readdir_internal(stream, infop, fatent); |
394 | ce->d_name = (d_names_start -= max_len); | 1553 | |
395 | snprintf(ce->d_name, max_len, VOL_NAMES, volume); | 1554 | int diridx = dirinfop->dcfile.idx; |
396 | dircache_size += max_len; | 1555 | unsigned int frontier = diridx < 0 ? |
397 | ce->info.attribute = FAT_ATTR_DIRECTORY | FAT_ATTR_VOLUME; | 1556 | dcvolp->frontier : get_entry(diridx)->frontier; |
398 | ce->info.size = 0; | 1557 | |
399 | append_position = dircache_gen_next(ce); | 1558 | int idx = infop->dcfile.idx; |
400 | ce = dircache_gen_down(ce); | 1559 | if (idx == 0) /* rewound? */ |
1560 | idx = diridx <= 0 ? dcvolp->root_down : get_entry(diridx)->down; | ||
1561 | else | ||
1562 | idx = get_entry(idx)->next; | ||
1563 | |||
1564 | struct dircache_entry *ce = get_entry(idx); | ||
1565 | if (frontier != FRONTIER_SETTLED) | ||
1566 | { | ||
1567 | /* the directory being read is reported to be incompletely cached; | ||
1568 | readthrough and if the entry exists, return it with its binding | ||
1569 | information; otherwise return the uncached read result while | ||
1570 | maintaining the last index */ | ||
1571 | int rc = uncached_readdir_internal(stream, infop, fatent); | ||
1572 | if (rc <= 0 || !ce || ce->direntry > infop->fatfile.e.entry) | ||
1573 | return rc; | ||
1574 | |||
1575 | /* entry matches next one to read */ | ||
1576 | } | ||
1577 | else if (!ce) | ||
1578 | { | ||
1579 | /* end of dir */ | ||
1580 | fat_empty_fat_direntry(fatent); | ||
1581 | infop->fatfile.e.entries = 0; | ||
1582 | return 0; | ||
401 | } | 1583 | } |
402 | #endif | ||
403 | 1584 | ||
404 | struct fat_direntry direntry; /* ditto */ | 1585 | /* FS entry information that we maintain */ |
405 | #ifdef HAVE_MULTIVOLUME | 1586 | entry_name_copy(fatent->name, ce); |
406 | sab.volume = volume; | 1587 | fatent->shortname[0] = '\0'; |
407 | #endif | 1588 | fatent->attr = ce->attr; |
408 | sab.dir = &sab_fat_dir; | 1589 | /* file code file scanning does not need time information */ |
409 | sab.direntry = &direntry; | 1590 | fatent->filesize = (ce->attr & ATTR_DIRECTORY) ? 0 : ce->filesize; |
410 | 1591 | fatent->firstcluster = ce->firstcluster; | |
411 | return sab_process_dir(0, ce); | 1592 | |
1593 | /* FS entry directory information */ | ||
1594 | infop->fatfile.e.entry = ce->direntry; | ||
1595 | infop->fatfile.e.entries = ce->direntries; | ||
1596 | |||
1597 | /* dircache file binding information */ | ||
1598 | infop->dcfile.idx = idx; | ||
1599 | infop->dcfile.serialnum = ce->serialnum; | ||
1600 | |||
1601 | /* return whether this needs decoding */ | ||
1602 | int rc = ce->direntries == 1 ? 2 : 1; | ||
1603 | |||
1604 | if (frontier == FRONTIER_SETTLED) | ||
1605 | { | ||
1606 | static long next_yield; | ||
1607 | if (TIME_AFTER(current_tick, next_yield)) | ||
1608 | { | ||
1609 | yield(); | ||
1610 | next_yield = current_tick + HZ/50; | ||
1611 | } | ||
1612 | } | ||
1613 | |||
1614 | return rc; | ||
412 | } | 1615 | } |
413 | #elif (CONFIG_PLATFORM & PLATFORM_HOSTED) /* PLATFORM_HOSTED */ | 1616 | |
1617 | /** | ||
1618 | * rewind the scan position for an internal scan | ||
1619 | */ | ||
1620 | void dircache_rewinddir_internal(struct file_base_info *infop) | ||
1621 | { | ||
1622 | uncached_rewinddir_internal(infop); | ||
1623 | dircache_dcfile_init(&infop->dcfile); | ||
1624 | } | ||
1625 | |||
1626 | #else /* !DIRCACHE_NATIVE (for all others) */ | ||
1627 | |||
1628 | ##################### | ||
1629 | /* we require access to the host functions */ | ||
1630 | #undef opendir | ||
1631 | #undef readdir | ||
1632 | #undef closedir | ||
1633 | #undef rewinddir | ||
1634 | |||
414 | static char sab_path[MAX_PATH]; | 1635 | static char sab_path[MAX_PATH]; |
415 | 1636 | ||
416 | static int sab_process_dir(struct dircache_entry *ce) | 1637 | static int sab_process_dir(struct dircache_entry *ce) |
417 | { | 1638 | { |
418 | struct dirent_uncached *entry; | 1639 | struct dirent_uncached *entry; |
419 | struct dircache_entry *first_ce = ce; | 1640 | struct dircache_entry *first_ce = ce; |
420 | DIR_UNCACHED *dir = opendir_uncached(sab_path); | 1641 | DIR *dir = opendir(sab_path); |
421 | if(dir == NULL) | 1642 | if(dir == NULL) |
422 | { | 1643 | { |
423 | logf("Failed to opendir_uncached(%s)", sab_path); | 1644 | logf("Failed to opendir_uncached(%s)", sab_path); |
424 | return -1; | 1645 | return -1; |
425 | } | 1646 | } |
426 | 1647 | ||
427 | while((entry = readdir_uncached(dir))) | 1648 | while (1) |
428 | { | 1649 | { |
429 | if(!strcmp(".", entry->d_name) || | 1650 | if (!(entry = readdir(dir))) |
430 | !strcmp("..", entry->d_name)) | 1651 | break; |
1652 | |||
1653 | if (IS_DOTDIR_NAME(entry->d_name)) | ||
1654 | { | ||
1655 | /* add "." and ".." */ | ||
1656 | ce->info.attribute = ATTR_DIRECTORY; | ||
1657 | ce->info.size = 0; | ||
1658 | ce->down = entry->d_name[1] == '\0' ? first_ce : first_ce->up; | ||
1659 | strcpy(ce->dot_d_name, entry->d_name); | ||
431 | continue; | 1660 | continue; |
1661 | } | ||
432 | 1662 | ||
433 | size_t size = strlen(entry->d_name) + 1; | 1663 | size_t size = strlen(entry->d_name) + 1; |
434 | ce->d_name = (d_names_start -= size); | 1664 | ce->d_name = (d_names_start -= size); |
@@ -436,10 +1666,10 @@ static int sab_process_dir(struct dircache_entry *ce) | |||
436 | 1666 | ||
437 | strcpy(ce->d_name, entry->d_name); | 1667 | strcpy(ce->d_name, entry->d_name); |
438 | dircache_size += size; | 1668 | dircache_size += size; |
439 | 1669 | ||
440 | if(entry->info.attribute & ATTR_DIRECTORY) | 1670 | if(entry->info.attribute & ATTR_DIRECTORY) |
441 | { | 1671 | { |
442 | dircache_gen_down(ce); | 1672 | dircache_gen_down(ce, ce); |
443 | if(ce->down == NULL) | 1673 | if(ce->down == NULL) |
444 | { | 1674 | { |
445 | closedir_uncached(dir); | 1675 | closedir_uncached(dir); |
@@ -450,1189 +1680,1504 @@ static int sab_process_dir(struct dircache_entry *ce) | |||
450 | /* append entry */ | 1680 | /* append entry */ |
451 | strlcpy(&sab_path[pathpos], "/", sizeof(sab_path) - pathpos); | 1681 | strlcpy(&sab_path[pathpos], "/", sizeof(sab_path) - pathpos); |
452 | strlcpy(&sab_path[pathpos+1], entry->d_name, sizeof(sab_path) - pathpos - 1); | 1682 | strlcpy(&sab_path[pathpos+1], entry->d_name, sizeof(sab_path) - pathpos - 1); |
453 | 1683 | ||
454 | int rc = sab_process_dir(ce->down); | 1684 | int rc = sab_process_dir(ce->down); |
455 | /* restore path */ | 1685 | /* restore path */ |
456 | sab_path[pathpos] = '\0'; | 1686 | sab_path[pathpos] = '\0'; |
457 | 1687 | ||
458 | if(rc < 0) | 1688 | if(rc < 0) |
459 | { | 1689 | { |
460 | closedir_uncached(dir); | 1690 | closedir_uncached(dir); |
461 | return rc; | 1691 | return rc; |
462 | } | 1692 | } |
463 | } | 1693 | } |
464 | 1694 | ||
465 | ce = dircache_gen_next(ce); | 1695 | ce = dircache_gen_entry(ce); |
466 | if(ce == NULL) | 1696 | if (ce == NULL) |
467 | return -5; | 1697 | return -5; |
468 | 1698 | ||
469 | /* When simulator is used, it's only safe to yield here. */ | 1699 | yield(); |
470 | if(thread_enabled) | ||
471 | { | ||
472 | /* Stop if we got an external signal. */ | ||
473 | if(check_event_queue()) | ||
474 | return -1; | ||
475 | yield(); | ||
476 | } | ||
477 | } | 1700 | } |
478 | 1701 | ||
479 | /* add "." and ".." */ | ||
480 | ce->d_name = dot; | ||
481 | ce->info.attribute = ATTR_DIRECTORY; | ||
482 | ce->info.size = 0; | ||
483 | ce->down = first_ce; | ||
484 | |||
485 | ce = dircache_gen_next(ce); | ||
486 | |||
487 | ce->d_name = dotdot; | ||
488 | ce->info.attribute = ATTR_DIRECTORY; | ||
489 | ce->info.size = 0; | ||
490 | ce->down = first_ce->up; | ||
491 | |||
492 | closedir_uncached(dir); | 1702 | closedir_uncached(dir); |
493 | return 0; | 1703 | return 1; |
494 | } | 1704 | } |
495 | 1705 | ||
496 | static int dircache_scan_and_build(IF_MV(int volume,) struct dircache_entry *ce) | 1706 | static int sab_process_volume(IF_MV(int volume,) struct dircache_entry *ce) |
497 | { | 1707 | { |
498 | #ifdef HAVE_MULTIVOLUME | ||
499 | (void) volume; | ||
500 | #endif | ||
501 | memset(ce, 0, sizeof(struct dircache_entry)); | 1708 | memset(ce, 0, sizeof(struct dircache_entry)); |
502 | |||
503 | strlcpy(sab_path, "/", sizeof sab_path); | 1709 | strlcpy(sab_path, "/", sizeof sab_path); |
504 | return sab_process_dir(ce); | 1710 | return sab_process_dir(ce); |
505 | } | 1711 | } |
506 | #endif /* PLATFORM_NATIVE */ | ||
507 | 1712 | ||
508 | /** | 1713 | int dircache_readdir_r(struct dircache_dirscan *dir, struct dirent *result) |
509 | * Internal function to get a pointer to dircache_entry for a given filename. | 1714 | { |
510 | * path: Absolute path to a file or directory (see comment) | 1715 | if (dircache_state != DIRCACHE_READY) |
511 | * go_down: Returns the first entry of the directory given by the path (see comment) | 1716 | return readdir_r(dir->###########3, result, &result); |
512 | * | ||
513 | * As a a special case, accept path="" as an alias for "/". | ||
514 | * Also if the path omits the first '/', it will be accepted. | ||
515 | * | ||
516 | * * If get_down=true: | ||
517 | * If path="/", the returned entry is the first of root directory (ie dircache_root) | ||
518 | * Otherwise, if 'entry' is the returned value when get_down=false, | ||
519 | * the functions returns entry->down (which can be NULL) | ||
520 | * | ||
521 | * * If get_down=false: | ||
522 | * If path="/chunk_1/chunk_2/.../chunk_n" then this functions returns the entry | ||
523 | * root_entry()->chunk_1->chunk_2->...->chunk_(n-1) | ||
524 | * Which means that | ||
525 | * dircache_get_entry(path)->d_name == chunk_n | ||
526 | * | ||
527 | * If path="/", the returned entry is NULL. | ||
528 | * If the entry doesn't exist, return NULL | ||
529 | * | ||
530 | * NOTE: this functions silently handles double '/' | ||
531 | */ | ||
532 | static struct dircache_entry* dircache_get_entry(const char *path, bool go_down) | ||
533 | { | ||
534 | char namecopy[MAX_PATH]; | ||
535 | char* part; | ||
536 | char* end; | ||
537 | |||
538 | bool at_root = true; | ||
539 | struct dircache_entry *cache_entry = dircache_root; | ||
540 | |||
541 | strlcpy(namecopy, path, sizeof(namecopy)); | ||
542 | |||
543 | for(part = strtok_r(namecopy, "/", &end); part; part = strtok_r(NULL, "/", &end)) | ||
544 | { | ||
545 | /* If request another chunk, the current entry has to be directory | ||
546 | * and so cache_entry->down has to be non-NULL/ | ||
547 | * Special case of root because it's already the first entry of the root directory | ||
548 | * | ||
549 | * NOTE: this is safe even if cache_entry->down is NULL */ | ||
550 | if(!at_root) | ||
551 | cache_entry = cache_entry->down; | ||
552 | else | ||
553 | at_root = false; | ||
554 | |||
555 | /* scan dir for name */ | ||
556 | while(cache_entry != NULL) | ||
557 | { | ||
558 | /* skip unused entries */ | ||
559 | if(cache_entry->d_name == NULL) | ||
560 | { | ||
561 | cache_entry = cache_entry->next; | ||
562 | continue; | ||
563 | } | ||
564 | /* compare names */ | ||
565 | if(!strcasecmp(part, cache_entry->d_name)) | ||
566 | break; | ||
567 | /* go to next entry */ | ||
568 | cache_entry = cache_entry->next; | ||
569 | } | ||
570 | |||
571 | /* handle not found case */ | ||
572 | if(cache_entry == NULL) | ||
573 | return NULL; | ||
574 | } | ||
575 | 1717 | ||
576 | /* NOTE: here cache_entry!=NULL so taking ->down is safe */ | 1718 | bool first = dir->dcinfo.scanidx == REWIND_INDEX; |
577 | if(go_down) | 1719 | struct dircache_entry *ce = get_entry(first ? dir->dcinfo.index : |
578 | return at_root ? cache_entry : cache_entry->down; | 1720 | dir->dcinfo.scanidx); |
579 | else | ||
580 | return at_root ? NULL : cache_entry; | ||
581 | } | ||
582 | 1721 | ||
583 | #ifdef HAVE_EEPROM_SETTINGS | 1722 | ce = first ? ce->down : ce->next; |
584 | 1723 | ||
585 | #define DIRCACHE_MAGIC 0x00d0c0a1 | 1724 | if (ce == NULL) |
586 | struct dircache_maindata { | 1725 | return 0; |
587 | long magic; | 1726 | |
588 | long size; | 1727 | dir->scanidx = ce - dircache_root; |
589 | long entry_count; | 1728 | |
590 | long appflags; | 1729 | strlcpy(result->d_name, ce->d_name, sizeof (result->d_name)); |
591 | struct dircache_entry *root_entry; | 1730 | result->info = ce->dirinfo; |
592 | char *d_names_start; | 1731 | |
593 | }; | 1732 | return 1; |
1733 | } | ||
1734 | |||
1735 | #endif /* DIRCACHE_* */ | ||
594 | 1736 | ||
595 | /** | 1737 | /** |
596 | * Function to load the internal cache structure from disk to initialize | 1738 | * reset the cache for the specified volume |
597 | * the dircache really fast and little disk access. | ||
598 | */ | 1739 | */ |
599 | int dircache_load(void) | 1740 | static void reset_volume(IF_MV_NONVOID(int volume)) |
600 | { | 1741 | { |
601 | struct dircache_maindata maindata; | 1742 | FOR_EACH_VOLUME(volume, i) |
602 | ssize_t bytes_read; | ||
603 | int fd; | ||
604 | |||
605 | if (dircache_initialized) | ||
606 | return -1; | ||
607 | |||
608 | logf("Loading directory cache"); | ||
609 | dircache_size = 0; | ||
610 | |||
611 | fd = open_dircache_file(O_RDONLY, 0); | ||
612 | if (fd < 0) | ||
613 | return -2; | ||
614 | |||
615 | bytes_read = read(fd, &maindata, sizeof(struct dircache_maindata)); | ||
616 | if (bytes_read != sizeof(struct dircache_maindata) | ||
617 | || maindata.magic != DIRCACHE_MAGIC || maindata.size <= 0) | ||
618 | { | 1743 | { |
619 | logf("Dircache file header error"); | 1744 | struct dircache_volume *dcvolp = DCVOL(i); |
620 | close(fd); | 1745 | |
621 | remove_dircache_file(); | 1746 | if (dcvolp->status == DIRCACHE_IDLE) |
622 | return -3; | 1747 | continue; /* idle => nothing happening there */ |
623 | } | 1748 | |
624 | 1749 | struct dircache_runinfo_volume *dcrivolp = DCRIVOL(i); | |
625 | allocated_size = maindata.size + DIRCACHE_RESERVE; | 1750 | |
626 | dircache_handle = core_alloc_ex("dircache", allocated_size, &ops); | 1751 | /* stop any scan and build on this one */ |
627 | /* block movement during upcoming I/O */ | 1752 | if (dcrivolp->sabp) |
628 | dont_move = true; | 1753 | dcrivolp->sabp->quit = true; |
629 | dircache_root = core_get_data(dircache_handle); | 1754 | |
630 | ALIGN_BUFFER(dircache_root, allocated_size, sizeof(struct dircache_entry*)); | 1755 | #ifdef HAVE_MULTIVOLUME |
631 | entry_count = maindata.entry_count; | 1756 | /* if this call is for all volumes, subsequent code will just reset |
632 | appflags = maindata.appflags; | 1757 | the cache memory usage and the freeing of individual entries may |
633 | 1758 | be skipped */ | |
634 | /* read the dircache file into memory, | 1759 | if (volume >= 0) |
635 | * start with the struct dircache_entries */ | 1760 | free_subentries(NULL, &dcvolp->root_down); |
636 | ssize_t bytes_to_read = entry_count*sizeof(struct dircache_entry); | 1761 | else |
637 | bytes_read = read(fd, dircache_root, bytes_to_read); | 1762 | #endif |
638 | 1763 | binding_dissolve_volume(dcrivolp); | |
639 | if (bytes_read != bytes_to_read) | 1764 | |
640 | { | 1765 | /* set it back to unscanned */ |
641 | logf("Dircache read failed #1"); | 1766 | dcvolp->status = DIRCACHE_IDLE; |
642 | return -6; | 1767 | dcvolp->frontier = FRONTIER_NEW; |
643 | } | 1768 | dcvolp->root_down = 0; |
644 | 1769 | dcvolp->build_ticks = 0; | |
645 | /* continue with the d_names. Fix up pointers to them if needed */ | 1770 | dcvolp->serialnum = 0; |
646 | bytes_to_read = maindata.size - bytes_to_read; | ||
647 | d_names_start = (char*)dircache_root + allocated_size - bytes_to_read; | ||
648 | bytes_read = read(fd, d_names_start, bytes_to_read); | ||
649 | close(fd); | ||
650 | remove_dircache_file(); | ||
651 | if (bytes_read != bytes_to_read) | ||
652 | { | ||
653 | logf("Dircache read failed #2"); | ||
654 | return -7; | ||
655 | } | 1771 | } |
1772 | } | ||
656 | 1773 | ||
657 | d_names_end = d_names_start + bytes_read; | 1774 | /** |
658 | dot = d_names_end - sizeof("."); | 1775 | * reset the entire cache state for all volumes |
659 | dotdot = dot - sizeof(".."); | 1776 | */ |
1777 | static void reset_cache(void) | ||
1778 | { | ||
1779 | if (!dircache_runinfo.handle) | ||
1780 | return; /* no buffer => nothing cached */ | ||
1781 | |||
1782 | /* blast all the volumes */ | ||
1783 | reset_volume(IF_MV(-1)); | ||
1784 | |||
1785 | #ifdef DIRCACHE_DUMPSTER | ||
1786 | dumpster_clean_buffer(dircache_runinfo.p + ENTRYSIZE, | ||
1787 | dircache_runinfo.bufsize); | ||
1788 | #endif /* DIRCACHE_DUMPSTER */ | ||
1789 | |||
1790 | /* reset the memory */ | ||
1791 | dircache.free_list = 0; | ||
1792 | dircache.size = 0; | ||
1793 | dircache.sizeused = 0; | ||
1794 | dircache.numentries = 0; | ||
1795 | dircache.names = dircache_runinfo.bufsize + ENTRYSIZE; | ||
1796 | dircache.sizenames = 0; | ||
1797 | dircache.namesfree = 0; | ||
1798 | dircache.nextnamefree = 0; | ||
1799 | *get_name(dircache.names - 1) = 0; | ||
1800 | /* dircache.last_serialnum stays */ | ||
1801 | /* dircache.reserve_used stays */ | ||
1802 | /* dircache.last_size stays */ | ||
1803 | } | ||
660 | 1804 | ||
661 | /* d_names are in reverse order, so the last entry points to the first string */ | 1805 | /** |
662 | ptrdiff_t offset_d_names = maindata.d_names_start - d_names_start; | 1806 | * checks each "idle" volume and builds it |
663 | ptrdiff_t offset_entries = maindata.root_entry - dircache_root; | 1807 | */ |
664 | offset_entries *= sizeof(struct dircache_entry); /* make it bytes */ | 1808 | static void build_volumes(void) |
1809 | { | ||
1810 | buffer_lock(); | ||
665 | 1811 | ||
666 | /* offset_entries is less likely to differ, so check if it's 0 in the loop | 1812 | for (int i = 0; i < NUM_VOLUMES; i++) |
667 | * offset_d_names however is almost always non-zero, since dircache_save() | ||
668 | * creates a file which causes the reserve buffer to be used. since | ||
669 | * we allocate a new, empty DIRCACHE_RESERVE here, the strings are | ||
670 | * farther behind */ | ||
671 | if (offset_entries != 0 || offset_d_names != 0) | ||
672 | { | 1813 | { |
673 | for(unsigned i = 0; i < entry_count; i++) | 1814 | /* this does reader locking but we already own that */ |
674 | { | 1815 | if (!volume_ismounted(IF_MV(i))) |
675 | if (dircache_root[i].d_name) | 1816 | continue; |
676 | dircache_root[i].d_name -= offset_d_names; | ||
677 | 1817 | ||
678 | if (offset_entries == 0) | 1818 | struct dircache_volume *dcvolp = DCVOL(i); |
679 | continue; | 1819 | |
680 | if (dircache_root[i].next_char) | 1820 | /* can't already be "scanning" because that's us; doesn't retry |
681 | dircache_root[i].next_char -= offset_entries; | 1821 | "ready" volumes */ |
682 | if (dircache_root[i].up_char) | 1822 | if (dcvolp->status == DIRCACHE_READY) |
683 | dircache_root[i].up_char -= offset_entries; | 1823 | continue; |
684 | if (dircache_root[i].down_char) | 1824 | |
685 | dircache_root[i].down_char -= offset_entries; | 1825 | /* measure how long it takes to build the cache for each volume */ |
686 | } | 1826 | if (!dcvolp->serialnum) |
1827 | dcvolp->serialnum = next_serialnum(); | ||
1828 | |||
1829 | dcvolp->status = DIRCACHE_SCANNING; | ||
1830 | dcvolp->start_tick = current_tick; | ||
1831 | |||
1832 | sab_process_volume(dcvolp); | ||
1833 | |||
1834 | if (dircache_runinfo.suspended) | ||
1835 | break; | ||
1836 | |||
1837 | /* whatever happened, it's ready unless reset */ | ||
1838 | dcvolp->build_ticks = current_tick - dcvolp->start_tick; | ||
1839 | dcvolp->status = DIRCACHE_READY; | ||
687 | } | 1840 | } |
688 | 1841 | ||
689 | /* Cache successfully loaded. */ | 1842 | size_t reserve_used = reserve_buf_used(); |
690 | dircache_size = maindata.size; | 1843 | if (reserve_used > dircache.reserve_used) |
691 | reserve_used = 0; | 1844 | dircache.reserve_used = reserve_used; |
692 | logf("Done, %ld KiB used", dircache_size / 1024); | 1845 | |
693 | dircache_initialized = true; | 1846 | if (DIRCACHE_STUFFED(reserve_used)) |
694 | memset(fd_bindings, 0, sizeof(fd_bindings)); | 1847 | dircache.last_size = 0; /* reset */ |
695 | dont_move = false; | 1848 | else if (dircache.size > dircache.last_size || |
1849 | dircache.last_size - dircache.size > DIRCACHE_RESERVE) | ||
1850 | dircache.last_size = dircache.size; | ||
1851 | |||
1852 | logf("Done, %ld KiB used", dircache.size / 1024); | ||
696 | 1853 | ||
697 | return 0; | 1854 | buffer_unlock(); |
698 | } | 1855 | } |
699 | 1856 | ||
700 | /** | 1857 | /** |
701 | * Function to save the internal cache stucture to disk for fast loading | 1858 | * allocate buffer and return whether or not a synchronous build should take |
702 | * on boot. | 1859 | * place; if 'realloced' is NULL, it's just a query about what will happen |
703 | */ | 1860 | */ |
704 | int dircache_save(void) | 1861 | static int prepare_build(bool *realloced) |
705 | { | 1862 | { |
706 | struct dircache_maindata maindata; | 1863 | /* called holding dircache lock */ |
707 | int fd; | 1864 | size_t size = dircache.last_size; |
708 | unsigned long bytes_written; | ||
709 | 1865 | ||
710 | remove_dircache_file(); | 1866 | #ifdef HAVE_EEPROM_SETTINGS |
711 | 1867 | if (realloced) | |
712 | if (!dircache_initialized) | 1868 | { |
713 | return -1; | 1869 | dircache_unlock(); |
1870 | remove_dircache_file(); | ||
1871 | dircache_lock(); | ||
714 | 1872 | ||
715 | logf("Saving directory cache"); | 1873 | if (dircache_runinfo.suspended) |
716 | dont_move = true; | 1874 | return -1; |
717 | fd = open_dircache_file(O_WRONLY | O_CREAT | O_TRUNC, 0666); | 1875 | } |
1876 | #endif /* HAVE_EEPROM_SETTINGS */ | ||
1877 | |||
1878 | bool stuffed = DIRCACHE_STUFFED(dircache.reserve_used); | ||
1879 | if (dircache_runinfo.bufsize > size && !stuffed) | ||
1880 | { | ||
1881 | if (realloced) | ||
1882 | *realloced = false; | ||
718 | 1883 | ||
719 | maindata.magic = DIRCACHE_MAGIC; | 1884 | return 0; /* start a transparent rebuild */ |
720 | maindata.size = dircache_size; | 1885 | } |
721 | maindata.root_entry = dircache_root; | ||
722 | maindata.d_names_start = d_names_start; | ||
723 | maindata.entry_count = entry_count; | ||
724 | maindata.appflags = appflags; | ||
725 | 1886 | ||
726 | /* Save the info structure */ | 1887 | int syncbuild = size > 0 && !stuffed ? 0 : 1; |
727 | bytes_written = write(fd, &maindata, sizeof(struct dircache_maindata)); | 1888 | |
728 | if (bytes_written != sizeof(struct dircache_maindata)) | 1889 | if (!realloced) |
1890 | return syncbuild; | ||
1891 | |||
1892 | if (syncbuild) | ||
729 | { | 1893 | { |
730 | close(fd); | 1894 | /* start a non-transparent rebuild */ |
731 | logf("dircache: write failed #1"); | 1895 | /* we'll use the entire audiobuf to allocate the dircache */ |
732 | return -2; | 1896 | size = audio_buffer_available() + dircache_runinfo.bufsize; |
1897 | /* try to allocate at least the min and no more than the limit */ | ||
1898 | size = MAX(DIRCACHE_MIN, MIN(size, DIRCACHE_LIMIT)); | ||
733 | } | 1899 | } |
1900 | else | ||
1901 | { | ||
1902 | /* start a transparent rebuild */ | ||
1903 | size = MAX(size, DIRCACHE_RESERVE) + DIRCACHE_RESERVE*2; | ||
1904 | } | ||
1905 | |||
1906 | *realloced = true; | ||
1907 | reset_cache(); | ||
734 | 1908 | ||
735 | /* Dump whole directory cache to disk | 1909 | buffer_lock(); |
736 | * start by writing the dircache_entries */ | 1910 | |
737 | size_t bytes_to_write = entry_count*sizeof(struct dircache_entry); | 1911 | int handle = reset_buffer(); |
738 | bytes_written = write(fd, dircache_root, bytes_to_write); | 1912 | dircache_unlock(); |
739 | if (bytes_written != bytes_to_write) | 1913 | |
1914 | if (handle > 0) | ||
1915 | core_free(handle); | ||
1916 | |||
1917 | handle = alloc_cache(size); | ||
1918 | |||
1919 | dircache_lock(); | ||
1920 | |||
1921 | if (dircache_runinfo.suspended && handle > 0) | ||
740 | { | 1922 | { |
741 | logf("dircache: write failed #2"); | 1923 | /* if we got suspended, don't keep this huge buffer around */ |
742 | return -3; | 1924 | dircache_unlock(); |
1925 | core_free(handle); | ||
1926 | handle = 0; | ||
1927 | dircache_lock(); | ||
743 | } | 1928 | } |
744 | 1929 | ||
745 | /* continue with the d_names */ | 1930 | if (handle <= 0) |
746 | bytes_to_write = d_names_end - d_names_start; | ||
747 | bytes_written = write(fd, d_names_start, bytes_to_write); | ||
748 | close(fd); | ||
749 | if (bytes_written != bytes_to_write) | ||
750 | { | 1931 | { |
751 | logf("dircache: write failed #3"); | 1932 | buffer_unlock(); |
752 | return -4; | 1933 | return -1; |
753 | } | 1934 | } |
754 | 1935 | ||
755 | dont_move = false; | 1936 | set_buffer(handle, size); |
756 | return 0; | 1937 | buffer_unlock(); |
1938 | |||
1939 | return syncbuild; | ||
757 | } | 1940 | } |
758 | #endif /* HAVE_EEPROM_SETTINGS */ | ||
759 | 1941 | ||
760 | /** | 1942 | /** |
761 | * Internal function which scans the disk and creates the dircache structure. | 1943 | * compact the dircache buffer after a successful full build |
762 | */ | 1944 | */ |
763 | static int dircache_do_rebuild(void) | 1945 | static void compact_cache(void) |
764 | { | 1946 | { |
765 | struct dircache_entry* root_entry; | 1947 | /* called holding dircache lock */ |
766 | unsigned int start_tick; | 1948 | if (dircache_runinfo.suspended) |
767 | int i; | 1949 | return; |
768 | |||
769 | /* Measure how long it takes build the cache. */ | ||
770 | start_tick = current_tick; | ||
771 | dircache_initializing = true; | ||
772 | appflags = 0; | ||
773 | 1950 | ||
774 | /* reset dircache and alloc root entry */ | 1951 | void *p = dircache_runinfo.p + ENTRYSIZE; |
775 | entry_count = 0; | 1952 | size_t leadsize = dircache.numentries * ENTRYSIZE + DIRCACHE_RESERVE; |
776 | root_entry = allocate_entry(); | ||
777 | dont_move = true; | ||
778 | 1953 | ||
779 | #ifdef HAVE_MULTIVOLUME | 1954 | void *dst = p + leadsize; |
780 | append_position = root_entry; | 1955 | void *src = get_name(dircache.names - 1); |
1956 | if (dst >= src) | ||
1957 | return; /* cache got bigger than expected; never mind that */ | ||
1958 | |||
1959 | /* slide the names up in memory */ | ||
1960 | memmove(dst, src, dircache.sizenames + 2); | ||
781 | 1961 | ||
782 | for (i = NUM_VOLUMES; i >= 0; i--) | 1962 | /* fix up name indexes */ |
1963 | ptrdiff_t offset = dst - src; | ||
1964 | |||
1965 | FOR_EACH_CACHE_ENTRY(ce) | ||
783 | { | 1966 | { |
784 | if (fat_ismounted(i)) | 1967 | if (!ce->tinyname) |
785 | { | 1968 | ce->name += offset; |
786 | #endif | ||
787 | cpu_boost(true); | ||
788 | #ifdef HAVE_MULTIVOLUME | ||
789 | if (dircache_scan_and_build(IF_MV(i,) append_position) < 0) | ||
790 | #else | ||
791 | if (dircache_scan_and_build(IF_MV(0,) root_entry) < 0) | ||
792 | #endif /* HAVE_MULTIVOLUME */ | ||
793 | { | ||
794 | logf("dircache_scan_and_build failed"); | ||
795 | cpu_boost(false); | ||
796 | dircache_size = 0; | ||
797 | dircache_initializing = false; | ||
798 | dont_move = false; | ||
799 | return -2; | ||
800 | } | ||
801 | cpu_boost(false); | ||
802 | #ifdef HAVE_MULTIVOLUME | ||
803 | } | ||
804 | } | 1969 | } |
805 | #endif | ||
806 | 1970 | ||
807 | logf("Done, %ld KiB used", dircache_size / 1024); | 1971 | dircache.names += offset; |
808 | |||
809 | dircache_initialized = true; | ||
810 | dircache_initializing = false; | ||
811 | cache_build_ticks = current_tick - start_tick; | ||
812 | |||
813 | /* Initialized fd bindings. */ | ||
814 | memset(fd_bindings, 0, sizeof(fd_bindings)); | ||
815 | for (i = 0; i < fdbind_idx; i++) | ||
816 | dircache_bind(fdbind_cache[i].fd, fdbind_cache[i].path); | ||
817 | fdbind_idx = 0; | ||
818 | |||
819 | if (thread_enabled) | ||
820 | { | ||
821 | if (allocated_size - dircache_size < DIRCACHE_RESERVE) | ||
822 | reserve_used = DIRCACHE_RESERVE - (allocated_size - dircache_size); | ||
823 | } | ||
824 | |||
825 | dont_move = false; | ||
826 | return 1; | ||
827 | } | ||
828 | 1972 | ||
829 | /* | 1973 | /* assumes beelzelib doesn't do things like calling callbacks or changing |
830 | * Free all associated resources, if any */ | 1974 | the pointer as a result of the shrink operation; it doesn't as of now |
831 | static void dircache_free(void) | 1975 | but if it ever does that may very well cause deadlock problems since |
832 | { | 1976 | we're holding filesystem locks */ |
833 | if (dircache_handle > 0) | 1977 | size_t newsize = leadsize + dircache.sizenames + 1; |
834 | dircache_handle = core_free(dircache_handle); | 1978 | core_shrink(dircache_runinfo.handle, p, newsize + 1); |
835 | dircache_size = allocated_size = 0; | 1979 | dircache_runinfo.bufsize = newsize; |
1980 | dircache.reserve_used = 0; | ||
836 | } | 1981 | } |
837 | 1982 | ||
838 | /** | 1983 | /** |
839 | * Internal thread that controls transparent cache building. | 1984 | * internal thread that controls cache building; exits when no more requests |
1985 | * are pending or the cache is suspended | ||
840 | */ | 1986 | */ |
841 | static void dircache_thread(void) | 1987 | static void dircache_thread(void) |
842 | { | 1988 | { |
843 | struct queue_event ev; | 1989 | struct queue_event ev; |
844 | 1990 | ||
1991 | /* calls made within the loop reopen the lock */ | ||
1992 | dircache_lock(); | ||
1993 | |||
845 | while (1) | 1994 | while (1) |
846 | { | 1995 | { |
847 | queue_wait(&dircache_queue, &ev); | 1996 | queue_wait_w_tmo(&dircache_queue, &ev, 0); |
848 | 1997 | if (ev.id == SYS_TIMEOUT || dircache_runinfo.suspended) | |
849 | switch (ev.id) | ||
850 | { | 1998 | { |
851 | #ifdef HAVE_HOTSWAP | 1999 | /* nothing left to do/suspended */ |
852 | case SYS_FS_CHANGED: | 2000 | if (dircache_runinfo.suspended) |
853 | if (!dircache_initialized) | 2001 | clear_dircache_queue(); |
854 | break; | 2002 | dircache_runinfo.thread_done = true; |
855 | dircache_initialized = false; | 2003 | break; |
856 | #endif | ||
857 | case DIRCACHE_BUILD: | ||
858 | thread_enabled = true; | ||
859 | if (dircache_do_rebuild() < 0) | ||
860 | dircache_free(); | ||
861 | thread_enabled = false; | ||
862 | break ; | ||
863 | |||
864 | case DIRCACHE_STOP: | ||
865 | logf("Stopped the rebuilding."); | ||
866 | dircache_initialized = false; | ||
867 | break ; | ||
868 | |||
869 | case SYS_USB_CONNECTED: | ||
870 | usb_acknowledge(SYS_USB_CONNECTED_ACK); | ||
871 | usb_wait_for_disconnect(&dircache_queue); | ||
872 | break ; | ||
873 | } | 2004 | } |
874 | } | ||
875 | } | ||
876 | 2005 | ||
877 | static void generate_dot_d_names(void) | 2006 | /* background-only builds are not allowed if a synchronous build is |
878 | { | 2007 | required first; test what needs to be done and if it checks out, |
879 | dot = (d_names_start -= sizeof(".")); | 2008 | do it for real below */ |
880 | dotdot = (d_names_start -= sizeof("..")); | 2009 | int *rcp = (int *)ev.data; |
881 | dircache_size += sizeof(".") + sizeof(".."); | 2010 | |
882 | strcpy(dot, "."); | 2011 | if (!rcp && prepare_build(NULL) > 0) |
883 | strcpy(dotdot, ".."); | 2012 | continue; |
2013 | |||
2014 | bool realloced; | ||
2015 | int rc = prepare_build(&realloced); | ||
2016 | if (rcp) | ||
2017 | *rcp = rc; | ||
2018 | |||
2019 | if (rc < 0) | ||
2020 | continue; | ||
2021 | |||
2022 | trigger_cpu_boost(); | ||
2023 | build_volumes(); | ||
2024 | |||
2025 | /* if it was reallocated, compact it */ | ||
2026 | if (realloced) | ||
2027 | compact_cache(); | ||
2028 | } | ||
2029 | |||
2030 | dircache_unlock(); | ||
884 | } | 2031 | } |
885 | 2032 | ||
886 | /** | 2033 | /** |
887 | * Start scanning the disk to build the dircache. | 2034 | * post a scan and build message to the thread, starting it if required |
888 | * Either transparent or non-transparent build method is used. | ||
889 | */ | 2035 | */ |
890 | int dircache_build(int last_size) | 2036 | static bool dircache_thread_post(int volatile *rcp) |
891 | { | 2037 | { |
892 | if (dircache_initialized || thread_enabled) | 2038 | if (dircache_runinfo.thread_done) |
893 | return -3; | 2039 | { |
2040 | /* mustn't recreate until it exits so that the stack isn't reused */ | ||
2041 | thread_wait(dircache_runinfo.thread_id); | ||
2042 | dircache_runinfo.thread_done = false; | ||
2043 | dircache_runinfo.thread_id = create_thread( | ||
2044 | dircache_thread, dircache_stack, sizeof (dircache_stack), 0, | ||
2045 | dircache_thread_name IF_PRIO(, PRIORITY_BACKGROUND) | ||
2046 | IF_COP(, CPU)); | ||
2047 | } | ||
894 | 2048 | ||
895 | logf("Building directory cache"); | 2049 | bool started = dircache_runinfo.thread_id != 0; |
896 | #ifdef HAVE_EEPROM_SETTINGS | ||
897 | remove_dircache_file(); | ||
898 | #endif | ||
899 | 2050 | ||
900 | /* Background build, dircache has been previously allocated and */ | 2051 | if (started) |
901 | if (allocated_size > MAX(last_size, 0)) | 2052 | queue_post(&dircache_queue, DCM_BUILD, (intptr_t)rcp); |
902 | { | ||
903 | d_names_start = d_names_end; | ||
904 | dircache_size = 0; | ||
905 | reserve_used = 0; | ||
906 | thread_enabled = true; | ||
907 | dircache_initializing = true; | ||
908 | generate_dot_d_names(); | ||
909 | |||
910 | queue_post(&dircache_queue, DIRCACHE_BUILD, 0); | ||
911 | return 2; | ||
912 | } | ||
913 | |||
914 | /* start by freeing, as we possibly re-allocate */ | ||
915 | dircache_free(); | ||
916 | |||
917 | if (last_size > DIRCACHE_RESERVE && last_size < DIRCACHE_LIMIT ) | ||
918 | { | ||
919 | allocated_size = last_size + DIRCACHE_RESERVE; | ||
920 | dircache_handle = core_alloc_ex("dircache", allocated_size, &ops); | ||
921 | dircache_root = core_get_data(dircache_handle); | ||
922 | ALIGN_BUFFER(dircache_root, allocated_size, sizeof(struct dircache_entry*)); | ||
923 | d_names_start = d_names_end = ((char*)dircache_root)+allocated_size-1; | ||
924 | dircache_size = 0; | ||
925 | thread_enabled = true; | ||
926 | generate_dot_d_names(); | ||
927 | |||
928 | /* Start a transparent rebuild. */ | ||
929 | queue_post(&dircache_queue, DIRCACHE_BUILD, 0); | ||
930 | return 3; | ||
931 | } | ||
932 | |||
933 | /* We'll use the entire audiobuf to allocate the dircache | ||
934 | * struct dircache_entrys are allocated from the beginning | ||
935 | * and their corresponding d_name from the end | ||
936 | * after generation the buffer will be compacted with DIRCACHE_RESERVE | ||
937 | * free bytes inbetween */ | ||
938 | size_t available = audio_buffer_available(); | ||
939 | /* try to allocate at least 1MB, the more the better */ | ||
940 | if (available < 1<<20) available = 1<<20; | ||
941 | if (available > DIRCACHE_LIMIT) available = DIRCACHE_LIMIT; | ||
942 | |||
943 | dircache_handle = core_alloc_ex("dircache", available, &ops); | ||
944 | if (dircache_handle <= 0) | ||
945 | return -1; /* that was not successful, should try rebooting */ | ||
946 | char* buf = core_get_data(dircache_handle); | ||
947 | dircache_root = (struct dircache_entry*)ALIGN_UP(buf, | ||
948 | sizeof(struct dircache_entry*)); | ||
949 | d_names_start = d_names_end = buf + available - 1; | ||
950 | dircache_size = 0; | ||
951 | generate_dot_d_names(); | ||
952 | |||
953 | /* Start a non-transparent rebuild. */ | ||
954 | int res = dircache_do_rebuild(); | ||
955 | if (res < 0) | ||
956 | goto fail; | ||
957 | |||
958 | /* now compact the dircache buffer */ | ||
959 | char* dst = ((char*)&dircache_root[entry_count] + DIRCACHE_RESERVE); | ||
960 | ptrdiff_t offset = d_names_start - dst; | ||
961 | if (offset <= 0) /* something went wrong */ | ||
962 | { | ||
963 | res = -1; | ||
964 | goto fail; | ||
965 | } | ||
966 | |||
967 | /* memmove d_names down, there's a possibility of overlap | ||
968 | * equivaent to dircache_size - entry_count*sizeof(struct dircache_entry) */ | ||
969 | ptrdiff_t size_to_move = d_names_end - d_names_start; | ||
970 | memmove(dst, d_names_start, size_to_move); | ||
971 | |||
972 | /* fix up pointers to the d_names */ | ||
973 | for(unsigned i = 0; i < entry_count; i++) | ||
974 | dircache_root[i].d_name -= offset; | ||
975 | |||
976 | d_names_start -= offset; | ||
977 | d_names_end -= offset; | ||
978 | dot -= offset; | ||
979 | dotdot -= offset; | ||
980 | |||
981 | /* equivalent to dircache_size + DIRCACHE_RESERVE + align */ | ||
982 | allocated_size = (d_names_end - buf); | ||
983 | reserve_used = 0; | ||
984 | |||
985 | core_shrink(dircache_handle, dircache_root, allocated_size); | ||
986 | return res; | ||
987 | fail: | ||
988 | dircache_disable(); | ||
989 | return res; | ||
990 | } | ||
991 | |||
992 | /** | ||
993 | * Main initialization function that must be called before any other | ||
994 | * operations within the dircache. | ||
995 | */ | ||
996 | void dircache_init(void) | ||
997 | { | ||
998 | int i; | ||
999 | int thread_id __attribute__((unused)); | ||
1000 | |||
1001 | dircache_initialized = false; | ||
1002 | dircache_initializing = false; | ||
1003 | |||
1004 | memset(opendirs, 0, sizeof(opendirs)); | ||
1005 | for (i = 0; i < MAX_OPEN_DIRS; i++) | ||
1006 | { | ||
1007 | opendirs[i].theent.d_name = opendir_dnames[i]; | ||
1008 | } | ||
1009 | |||
1010 | queue_init(&dircache_queue, true); | ||
1011 | thread_id = create_thread(dircache_thread, dircache_stack, | ||
1012 | sizeof(dircache_stack), 0, dircache_thread_name | ||
1013 | IF_PRIO(, PRIORITY_BACKGROUND) | ||
1014 | IF_COP(, CPU)); | ||
1015 | #ifdef HAVE_IO_PRIORITY | ||
1016 | thread_set_io_priority(thread_id,IO_PRIORITY_BACKGROUND); | ||
1017 | #endif | ||
1018 | 2053 | ||
2054 | return started; | ||
1019 | } | 2055 | } |
1020 | 2056 | ||
1021 | /** | 2057 | /** |
1022 | * Returns true if dircache has been initialized and is ready to be used. | 2058 | * wait for the dircache thread to finish building; intended for waiting for a |
2059 | * non-transparent build to finish when dircache_resume() returns > 0 | ||
1023 | */ | 2060 | */ |
1024 | bool dircache_is_enabled(void) | 2061 | void dircache_wait(void) |
1025 | { | 2062 | { |
1026 | return dircache_initialized; | 2063 | thread_wait(dircache_runinfo.thread_id); |
1027 | } | 2064 | } |
1028 | 2065 | ||
1029 | /** | 2066 | /** |
1030 | * Returns true if dircache is being initialized. | 2067 | * call after mounting a volume or all volumes |
1031 | */ | 2068 | */ |
1032 | bool dircache_is_initializing(void) | 2069 | void dircache_mount(void) |
1033 | { | 2070 | { |
1034 | return dircache_initializing || thread_enabled; | 2071 | /* call with writer exclusion */ |
2072 | if (dircache_runinfo.suspended) | ||
2073 | return; | ||
2074 | |||
2075 | dircache_thread_post(NULL); | ||
1035 | } | 2076 | } |
1036 | 2077 | ||
1037 | /** | 2078 | /** |
1038 | * Set application flags used to determine if dircache is still intact. | 2079 | * call after unmounting a volume; specifying < 0 for all or >= 0 for the |
2080 | * specific one | ||
1039 | */ | 2081 | */ |
1040 | void dircache_set_appflag(long mask) | 2082 | void dircache_unmount(IF_MV_NONVOID(int volume)) |
1041 | { | 2083 | { |
1042 | appflags |= mask; | 2084 | /* call with writer exclusion */ |
2085 | if (dircache_runinfo.suspended) | ||
2086 | return; | ||
2087 | |||
2088 | #ifdef HAVE_MULTIVOLUME | ||
2089 | if (volume >= 0) | ||
2090 | reset_volume(volume); | ||
2091 | else | ||
2092 | #endif /* HAVE_MULTIVOLUME */ | ||
2093 | reset_cache(); | ||
2094 | } | ||
2095 | |||
2096 | /* backend to dircache_suspend() and dircache_disable() */ | ||
2097 | static void dircache_suspend_internal(bool freeit) | ||
2098 | { | ||
2099 | if (dircache_runinfo.suspended++ > 0 && | ||
2100 | (!freeit || dircache_runinfo.handle <= 0)) | ||
2101 | return; | ||
2102 | |||
2103 | unsigned int thread_id = dircache_runinfo.thread_id; | ||
2104 | |||
2105 | reset_cache(); | ||
2106 | clear_dircache_queue(); | ||
2107 | |||
2108 | /* grab the buffer away into our control; the cache won't need it now */ | ||
2109 | int handle = 0; | ||
2110 | if (freeit) | ||
2111 | handle = reset_buffer(); | ||
2112 | |||
2113 | dircache_unlock(); | ||
2114 | |||
2115 | if (handle > 0) | ||
2116 | core_free(handle); | ||
2117 | |||
2118 | thread_wait(thread_id); | ||
2119 | |||
2120 | dircache_lock(); | ||
2121 | } | ||
2122 | |||
2123 | /* backend to dircache_resume() and dircache_enable() */ | ||
2124 | static int dircache_resume_internal(bool build_now) | ||
2125 | { | ||
2126 | int volatile rc = 0; | ||
2127 | |||
2128 | if (dircache_runinfo.suspended == 0 || --dircache_runinfo.suspended == 0) | ||
2129 | rc = build_now && dircache_runinfo.enabled ? 1 : 0; | ||
2130 | |||
2131 | if (rc) | ||
2132 | rc = dircache_thread_post(&rc) ? INT_MIN : -1; | ||
2133 | |||
2134 | if (rc == INT_MIN) | ||
2135 | { | ||
2136 | dircache_unlock(); | ||
2137 | |||
2138 | while (rc == INT_MIN) /* poll for response */ | ||
2139 | sleep(0); | ||
2140 | |||
2141 | dircache_lock(); | ||
2142 | } | ||
2143 | |||
2144 | return rc < 0 ? rc * 10 - 2 : rc; | ||
1043 | } | 2145 | } |
1044 | 2146 | ||
1045 | /** | 2147 | /** |
1046 | * Get application flags used to determine if dircache is still intact. | 2148 | * service to dircache_enable() and dircache_load(); "build_now" starts a build |
2149 | * immediately if the cache was not enabled | ||
1047 | */ | 2150 | */ |
1048 | bool dircache_get_appflag(long mask) | 2151 | static int dircache_enable_internal(bool build_now) |
1049 | { | 2152 | { |
1050 | return dircache_is_enabled() && (appflags & mask); | 2153 | int rc = 0; |
2154 | |||
2155 | if (!dircache_runinfo.enabled) | ||
2156 | { | ||
2157 | dircache_runinfo.enabled = true; | ||
2158 | rc = dircache_resume_internal(build_now); | ||
2159 | } | ||
2160 | |||
2161 | return rc; | ||
1051 | } | 2162 | } |
1052 | 2163 | ||
1053 | /** | 2164 | /** |
1054 | * Returns the current number of entries (directories and files) in the cache. | 2165 | * service to dircache_disable() |
1055 | */ | 2166 | */ |
1056 | int dircache_get_entry_count(void) | 2167 | static void dircache_disable_internal(void) |
1057 | { | 2168 | { |
1058 | return entry_count; | 2169 | if (dircache_runinfo.enabled) |
2170 | { | ||
2171 | dircache_runinfo.enabled = false; | ||
2172 | dircache_suspend_internal(true); | ||
2173 | } | ||
1059 | } | 2174 | } |
1060 | 2175 | ||
1061 | /** | 2176 | /** |
1062 | * Returns the allocated space for dircache (without reserve space). | 2177 | * disables dircache without freeing the buffer (so it can be re-enabled |
2178 | * afterwards with dircache_resume(); usually called when accepting an USB | ||
2179 | * connection | ||
1063 | */ | 2180 | */ |
1064 | int dircache_get_cache_size(void) | 2181 | void dircache_suspend(void) |
1065 | { | 2182 | { |
1066 | return dircache_is_enabled() ? dircache_size : 0; | 2183 | dircache_lock(); |
2184 | dircache_suspend_internal(false); | ||
2185 | dircache_unlock(); | ||
1067 | } | 2186 | } |
1068 | 2187 | ||
1069 | /** | 2188 | /** |
1070 | * Returns how many bytes of the reserve allocation for live cache | 2189 | * re-enables the dircache if previously suspended by dircache_suspend |
1071 | * updates have been used. | 2190 | * or dircache_steal_buffer(), re-using the already allocated buffer if |
2191 | * available | ||
2192 | * | ||
2193 | * returns: 0 if the background build is started or dircache is still | ||
2194 | * suspended | ||
2195 | * > 0 if the build is non-background | ||
2196 | * < 0 upon failure | ||
1072 | */ | 2197 | */ |
1073 | int dircache_get_reserve_used(void) | 2198 | int dircache_resume(void) |
1074 | { | 2199 | { |
1075 | return dircache_is_enabled() ? reserve_used : 0; | 2200 | dircache_lock(); |
2201 | int rc = dircache_resume_internal(true); | ||
2202 | dircache_unlock(); | ||
2203 | return rc; | ||
1076 | } | 2204 | } |
1077 | 2205 | ||
1078 | /** | 2206 | /** |
1079 | * Returns the time in kernel ticks that took to build the cache. | 2207 | * as dircache_resume() but globally enables it; called by settings and init |
1080 | */ | 2208 | */ |
1081 | int dircache_get_build_ticks(void) | 2209 | int dircache_enable(void) |
1082 | { | 2210 | { |
1083 | return dircache_is_enabled() ? cache_build_ticks : 0; | 2211 | dircache_lock(); |
2212 | int rc = dircache_enable_internal(true); | ||
2213 | dircache_unlock(); | ||
2214 | return rc; | ||
1084 | } | 2215 | } |
1085 | 2216 | ||
1086 | /** | 2217 | /** |
1087 | * Disables dircache without freeing the buffer (so it can be re-enabled | 2218 | * as dircache_suspend() but also frees the buffer; usually called on shutdown |
1088 | * afterwards with dircache_resume() or dircache_build()), usually | 2219 | * or when deactivated |
1089 | * called when accepting an usb connection */ | 2220 | */ |
1090 | void dircache_suspend(void) | 2221 | void dircache_disable(void) |
1091 | { | 2222 | { |
1092 | int i; | 2223 | dircache_lock(); |
1093 | bool cache_in_use; | 2224 | dircache_disable_internal(); |
1094 | 2225 | dircache_unlock(); | |
1095 | if (thread_enabled) | ||
1096 | queue_post(&dircache_queue, DIRCACHE_STOP, 0); | ||
1097 | |||
1098 | while (thread_enabled) | ||
1099 | sleep(1); | ||
1100 | dircache_initialized = false; | ||
1101 | |||
1102 | logf("Waiting for cached dirs to release"); | ||
1103 | do { | ||
1104 | cache_in_use = false; | ||
1105 | for (i = 0; i < MAX_OPEN_DIRS; i++) { | ||
1106 | if (!opendirs[i].regulardir && opendirs[i].busy) | ||
1107 | { | ||
1108 | cache_in_use = true; | ||
1109 | sleep(1); | ||
1110 | break ; | ||
1111 | } | ||
1112 | } | ||
1113 | } while (cache_in_use) ; | ||
1114 | |||
1115 | logf("Cache released"); | ||
1116 | entry_count = 0; | ||
1117 | } | 2226 | } |
1118 | 2227 | ||
1119 | /** | 2228 | /** |
1120 | * Re-enables the dircache if previous suspended by dircache_suspend | 2229 | * have dircache give up its allocation; call dircache_resume() to restart it |
1121 | * or dircache_steal_buffer(), re-using the already allocated buffer | ||
1122 | * | ||
1123 | * Returns true if the background build is started, false otherwise | ||
1124 | * (e.g. if no buffer was previously allocated) | ||
1125 | */ | 2230 | */ |
1126 | bool dircache_resume(void) | 2231 | void dircache_free_buffer(void) |
1127 | { | 2232 | { |
1128 | bool ret = allocated_size > 0; | 2233 | dircache_lock(); |
1129 | if (ret) /* only resume if already allocated */ | 2234 | dircache_suspend_internal(true); |
1130 | ret = (dircache_build(0) > 0); | 2235 | dircache_unlock(); |
2236 | } | ||
2237 | |||
2238 | |||
2239 | /** Dircache live updating **/ | ||
1131 | 2240 | ||
1132 | return (allocated_size > 0); | 2241 | /** |
2242 | * obtain binding information for the file's root volume; this is the starting | ||
2243 | * point for internal path parsing and binding | ||
2244 | */ | ||
2245 | void dircache_get_rootinfo(struct file_base_info *infop) | ||
2246 | { | ||
2247 | int volume = BASEINFO_VOL(infop); | ||
2248 | struct dircache_volume *dcvolp = DCVOL(volume); | ||
2249 | |||
2250 | if (dcvolp->serialnum) | ||
2251 | { | ||
2252 | /* root has a binding */ | ||
2253 | infop->dcfile.idx = -volume - 1; | ||
2254 | infop->dcfile.serialnum = dcvolp->serialnum; | ||
2255 | } | ||
2256 | else | ||
2257 | { | ||
2258 | /* root is idle */ | ||
2259 | dircache_dcfile_init(&infop->dcfile); | ||
2260 | } | ||
1133 | } | 2261 | } |
1134 | 2262 | ||
1135 | /** | 2263 | /** |
1136 | * Disables the dircache entirely. Usually called on shutdown or when | 2264 | * called by file code when the first reference to a file or directory is |
1137 | * deactivated | 2265 | * opened |
1138 | */ | 2266 | */ |
1139 | void dircache_disable(void) | 2267 | void dircache_bind_file(struct file_base_binding *bindp) |
2268 | { | ||
2269 | /* requires write exclusion */ | ||
2270 | logf("dc open: %u", (unsigned int)bindp->info.dcfile.serialnum); | ||
2271 | binding_open(bindp); | ||
2272 | } | ||
2273 | |||
2274 | /** | ||
2275 | * called by file code when the last reference to a file or directory is | ||
2276 | * closed | ||
2277 | */ | ||
2278 | void dircache_unbind_file(struct file_base_binding *bindp) | ||
1140 | { | 2279 | { |
1141 | dircache_suspend(); | 2280 | /* requires write exclusion */ |
1142 | dircache_free(); | 2281 | logf("dc close: %u", (unsigned int)bindp->info.dcfile.serialnum); |
2282 | binding_close(bindp); | ||
1143 | } | 2283 | } |
1144 | 2284 | ||
1145 | /** | 2285 | /** |
1146 | * Steal the allocated dircache buffer and disable dircache. | 2286 | * called by file code when a file is newly created |
1147 | */ | 2287 | */ |
1148 | void* dircache_steal_buffer(size_t *size) | 2288 | void dircache_fileop_create(struct file_base_info *dirinfop, |
2289 | struct file_base_binding *bindp, | ||
2290 | const char *basename, | ||
2291 | const struct dirinfo_native *dinp) | ||
1149 | { | 2292 | { |
1150 | dircache_suspend(); | 2293 | /* requires write exclusion */ |
1151 | if (dircache_size == 0) | 2294 | logf("dc create: %u \"%s\"", |
2295 | (unsigned int)bindp->info.dcfile.serialnum, basename); | ||
2296 | |||
2297 | if (!dirinfop->dcfile.serialnum) | ||
1152 | { | 2298 | { |
1153 | *size = 0; | 2299 | /* no parent binding => no child binding */ |
1154 | return NULL; | 2300 | return; |
1155 | } | 2301 | } |
1156 | 2302 | ||
1157 | /* since we give up the buffer (without freeing), it must not move anymore */ | 2303 | struct dircache_entry *ce; |
1158 | dont_move = true; | 2304 | int idx = create_entry(basename, &ce); |
1159 | *size = dircache_size + (DIRCACHE_RESERVE-reserve_used); | 2305 | if (idx == 0) |
1160 | 2306 | { | |
1161 | return dircache_root; | 2307 | /* failed allocation; parent cache contents are not complete */ |
2308 | establish_frontier(dirinfop->dcfile.idx, FRONTIER_ZONED); | ||
2309 | return; | ||
2310 | } | ||
2311 | |||
2312 | struct file_base_info *infop = &bindp->info; | ||
2313 | |||
2314 | #ifdef DIRCACHE_NATIVE | ||
2315 | ce->firstcluster = infop->fatfile.firstcluster; | ||
2316 | ce->direntry = infop->fatfile.e.entry; | ||
2317 | ce->direntries = infop->fatfile.e.entries; | ||
2318 | ce->wrtdate = dinp->wrtdate; | ||
2319 | ce->wrttime = dinp->wrttime; | ||
2320 | #else | ||
2321 | ce->mtime = dinp->mtime; | ||
2322 | #endif | ||
2323 | ce->attr = dinp->attr; | ||
2324 | if (!(dinp->attr & ATTR_DIRECTORY)) | ||
2325 | ce->filesize = dinp->size; | ||
2326 | |||
2327 | insert_file_entry(dirinfop, ce); | ||
2328 | |||
2329 | /* file binding will have been queued when it was opened; just resolve */ | ||
2330 | infop->dcfile.idx = idx; | ||
2331 | infop->dcfile.serialnum = ce->serialnum; | ||
2332 | binding_resolve(infop); | ||
2333 | |||
2334 | if ((dinp->attr & ATTR_DIRECTORY) && !is_dotdir_name(basename)) | ||
2335 | { | ||
2336 | /* scan-in the contents of the new directory at this level only */ | ||
2337 | buffer_lock(); | ||
2338 | sab_process_dir(infop, false); | ||
2339 | buffer_unlock(); | ||
2340 | } | ||
1162 | } | 2341 | } |
1163 | 2342 | ||
1164 | /** | 2343 | /** |
1165 | * Usermode function to return dircache_entry index to the given path. | 2344 | * called by file code when a file or directory is removed |
1166 | */ | 2345 | */ |
1167 | static int dircache_get_entry_id_ex(const char *filename, bool go_down) | 2346 | void dircache_fileop_remove(struct file_base_binding *bindp) |
1168 | { | 2347 | { |
1169 | if (!dircache_initialized || filename == NULL) | 2348 | /* requires write exclusion */ |
1170 | return -1; | 2349 | logf("dc remove: %u\n", (unsigned int)bindp->info.dcfile.serialnum); |
1171 | |||
1172 | struct dircache_entry* res = dircache_get_entry(filename, go_down); | ||
1173 | return res ? res - dircache_root : -1; | ||
1174 | } | ||
1175 | 2350 | ||
1176 | int dircache_get_entry_id(const char* filename) | 2351 | if (!bindp->info.dcfile.serialnum) |
1177 | { | 2352 | return; /* no binding yet */ |
1178 | return dircache_get_entry_id_ex(filename, false); | 2353 | |
2354 | free_file_entry(&bindp->info); | ||
2355 | |||
2356 | /* if binding was resolved; it should now be queued via above call */ | ||
1179 | } | 2357 | } |
1180 | 2358 | ||
1181 | /** | 2359 | /** |
1182 | * Internal: Get the startcluster for the index | 2360 | * called by file code when a file is renamed |
1183 | */ | 2361 | */ |
1184 | long _dircache_get_entry_startcluster(int id) | 2362 | void dircache_fileop_rename(struct file_base_info *dirinfop, |
2363 | struct file_base_binding *bindp, | ||
2364 | const char *basename) | ||
1185 | { | 2365 | { |
1186 | return get_entry(id)->startcluster; | 2366 | /* requires write exclusion */ |
2367 | logf("dc rename: %u \"%s\"", | ||
2368 | (unsigned int)bindp->info.dcfile.serialnum, basename); | ||
2369 | |||
2370 | if (!dirinfop->dcfile.serialnum) | ||
2371 | { | ||
2372 | /* new parent directory not cached; there is nowhere to put it so | ||
2373 | nuke it */ | ||
2374 | if (bindp->info.dcfile.serialnum) | ||
2375 | free_file_entry(&bindp->info); | ||
2376 | /* else no entry anyway */ | ||
2377 | |||
2378 | return; | ||
2379 | } | ||
2380 | |||
2381 | if (!bindp->info.dcfile.serialnum) | ||
2382 | { | ||
2383 | /* binding not resolved on the old file but it's going into a resolved | ||
2384 | parent which means the parent would be missing an entry in the cache; | ||
2385 | downgrade the parent */ | ||
2386 | establish_frontier(dirinfop->dcfile.idx, FRONTIER_ZONED); | ||
2387 | return; | ||
2388 | } | ||
2389 | |||
2390 | /* unlink the entry but keep it; it needs to be re-sorted since the | ||
2391 | underlying FS probably changed the order */ | ||
2392 | struct dircache_entry *ce = remove_file_entry(&bindp->info); | ||
2393 | |||
2394 | #ifdef DIRCACHE_NATIVE | ||
2395 | /* update other name-related information before inserting */ | ||
2396 | ce->direntry = bindp->info.fatfile.e.entry; | ||
2397 | ce->direntries = bindp->info.fatfile.e.entries; | ||
2398 | #endif | ||
2399 | |||
2400 | /* place it into its new home */ | ||
2401 | insert_file_entry(dirinfop, ce); | ||
2402 | |||
2403 | /* lastly, update the entry name itself */ | ||
2404 | if (entry_reassign_name(ce, basename)) | ||
2405 | { | ||
2406 | /* it's not really the same one now so re-stamp it */ | ||
2407 | dc_serial_t serialnum = next_serialnum(); | ||
2408 | ce->serialnum = serialnum; | ||
2409 | bindp->info.dcfile.serialnum = serialnum; | ||
2410 | } | ||
2411 | else | ||
2412 | { | ||
2413 | /* it cannot be kept around without a valid name */ | ||
2414 | free_file_entry(&bindp->info); | ||
2415 | establish_frontier(dirinfop->dcfile.idx, FRONTIER_ZONED); | ||
2416 | } | ||
1187 | } | 2417 | } |
1188 | 2418 | ||
1189 | /** | 2419 | /** |
1190 | * Internal: Get the struct dirinfo for the index | 2420 | * called by file code to synchronize file entry information |
1191 | */ | 2421 | */ |
1192 | struct dirinfo* _dircache_get_entry_dirinfo(int id) | 2422 | void dircache_fileop_sync(struct file_base_binding *bindp, |
2423 | const struct dirinfo_native *dinp) | ||
1193 | { | 2424 | { |
1194 | return &get_entry(id)->info; | 2425 | /* requires write exclusion */ |
2426 | struct file_base_info *infop = &bindp->info; | ||
2427 | logf("dc sync: %u\n", (unsigned int)infop->dcfile.serialnum); | ||
2428 | |||
2429 | if (!infop->dcfile.serialnum) | ||
2430 | return; /* binding unresolved */ | ||
2431 | |||
2432 | struct dircache_entry *ce = get_entry(infop->dcfile.idx); | ||
2433 | if (!ce) | ||
2434 | { | ||
2435 | logf(" bad index %d", infop->dcfile.idx); | ||
2436 | return; /* a root (should never be called for this) */ | ||
2437 | } | ||
2438 | |||
2439 | #ifdef DIRCACHE_NATIVE | ||
2440 | ce->firstcluster = infop->fatfile.firstcluster; | ||
2441 | ce->wrtdate = dinp->wrtdate; | ||
2442 | ce->wrttime = dinp->wrttime; | ||
2443 | #else | ||
2444 | ce->mtime = dinp->mtime; | ||
2445 | #endif | ||
2446 | ce->attr = dinp->attr; | ||
2447 | if (!(dinp->attr & ATTR_DIRECTORY)) | ||
2448 | ce->filesize = dinp->size; | ||
1195 | } | 2449 | } |
1196 | 2450 | ||
1197 | /* | 2451 | |
1198 | * build a path from an entry upto the root using recursion | 2452 | /** Dircache paths and files **/ |
1199 | * | 2453 | |
1200 | * it appends '/' after strlcat, therefore buf[0] needs to be prepared with '/' | 2454 | #ifdef DIRCACHE_DUMPSTER |
1201 | * and it will leave a trailing '/' | 2455 | /* helper for dircache_get_path() */ |
1202 | * | 2456 | static ssize_t get_path_sub(int idx, char *buf, size_t size) |
1203 | * returns the position of that trailing '/' so it can be deleted afterwards | ||
1204 | * (or, in case of truncation, the position of the nul byte */ | ||
1205 | static size_t copy_path_helper(const struct dircache_entry *entry, char *buf, size_t size) | ||
1206 | { | 2457 | { |
1207 | int offset = 1; | 2458 | if (idx == 0) |
1208 | /* has parent? */ | 2459 | return -2; /* entry is an orphan split from any root */ |
1209 | if (entry->up) | ||
1210 | offset += copy_path_helper(entry->up, buf, size); | ||
1211 | 2460 | ||
1212 | size_t len = strlcpy(buf+offset, entry->d_name, size - offset) + offset; | 2461 | ssize_t offset; |
1213 | if (len < size) | 2462 | char *cename; |
2463 | |||
2464 | if (idx > 0) | ||
1214 | { | 2465 | { |
1215 | buf[len++] = '/'; | 2466 | /* go all the way up then move back down from the root */ |
1216 | buf[len] = '\0'; | 2467 | struct dircache_entry *ce = get_entry(idx); |
2468 | offset = get_path_sub(ce->up, buf, size) - 1; | ||
2469 | if (offset < 0) | ||
2470 | return -3; | ||
2471 | |||
2472 | cename = alloca(MAX_NAME + 1); | ||
2473 | entry_name_copy(cename, ce); | ||
1217 | } | 2474 | } |
1218 | return len-1; | 2475 | else /* idx < 0 */ |
2476 | { | ||
2477 | offset = 0; | ||
2478 | cename = ""; | ||
2479 | |||
2480 | #ifdef HAVE_MULTIVOLUME | ||
2481 | int volume = IF_MV_VOL(-idx - 1); | ||
2482 | if (volume > 0) | ||
2483 | { | ||
2484 | /* prepend the volume specifier for volumes > 0 */ | ||
2485 | cename = alloca(VOL_MAX_LEN+1); | ||
2486 | get_volume_name(volume, cename); | ||
2487 | } | ||
2488 | #endif /* HAVE_MULTIVOLUME */ | ||
2489 | } | ||
2490 | |||
2491 | return offset + path_append(buf + offset, PA_SEP_HARD, cename, | ||
2492 | size > (size_t)offset ? size - offset : 0); | ||
1219 | } | 2493 | } |
2494 | #endif /* DIRCACHE_DUMPSTER */ | ||
2495 | |||
2496 | #if 0 | ||
2497 | |||
1220 | /** | 2498 | /** |
1221 | * Function to copy the full absolute path from dircache to the given buffer | 2499 | * retrieve and validate the file's entry/binding serial number |
1222 | * using the given dircache_entry pointer. | 2500 | * the dircache file's serial number must match the indexed entry's or the |
1223 | * | 2501 | * file reference is stale |
1224 | * Returns the size of the resulting string, or 0 if an error occured | ||
1225 | */ | 2502 | */ |
1226 | size_t dircache_copy_path(int index, char *buf, size_t size) | 2503 | static dc_serial_t get_file_serialnum(const struct dircache_file *dcfilep) |
1227 | { | 2504 | { |
1228 | if (!size || !buf || index < 0) | 2505 | int idx = dcfilep->idx; |
2506 | |||
2507 | if (idx == 0 || idx < -NUM_VOLUMES) | ||
1229 | return 0; | 2508 | return 0; |
1230 | 2509 | ||
1231 | buf[0] = '/'; | 2510 | dc_serial_t serialnum; |
1232 | size_t res = copy_path_helper(&dircache_root[index], buf, size - 1); | 2511 | |
1233 | /* fixup trailing '/' */ | 2512 | if (idx > 0) |
1234 | buf[res] = '\0'; | 2513 | { |
1235 | return res; | 2514 | struct dircache_entry *ce = get_entry(idx); |
2515 | serialnum = ce ? ce->serialnum : 0; | ||
2516 | } | ||
2517 | else | ||
2518 | { | ||
2519 | serialnum = get_idx_dcvolp(idx)->serialnum; | ||
2520 | } | ||
2521 | |||
2522 | return serialnum == dcfilep->serialnum ? serialnum : 0; | ||
1236 | } | 2523 | } |
1237 | 2524 | ||
1238 | /* --- Directory cache live updating functions --- */ | 2525 | /** |
1239 | static int block_until_ready(void) | 2526 | * usermode function to construct a full absolute path from dircache into the |
2527 | * given buffer given the dircache file info | ||
2528 | * | ||
2529 | * returns: | ||
2530 | * success - the length of the string, not including the trailing null | ||
2531 | * failure - a negative value | ||
2532 | * | ||
2533 | * successful return value is as strlcpy() | ||
2534 | * | ||
2535 | * errors: | ||
2536 | * ENOENT - the file or directory does not exist | ||
2537 | */ | ||
2538 | ssize_t dircache_get_path(const struct dircache_file *dcfilep, char *buf, | ||
2539 | size_t size) | ||
1240 | { | 2540 | { |
1241 | /* Block until dircache has been built. */ | 2541 | /* if missing buffer space, still return what's needed a la strlcpy */ |
1242 | while (!dircache_initialized && dircache_is_initializing()) | 2542 | if (!buf) |
1243 | sleep(1); | 2543 | size = 0; |
1244 | 2544 | else if (size) | |
1245 | if (!dircache_initialized) | 2545 | *buf = '\0'; |
1246 | return -1; | 2546 | |
1247 | 2547 | ssize_t len = -1; | |
1248 | return 0; | 2548 | |
2549 | dircache_lock(); | ||
2550 | |||
2551 | /* first and foremost, there must be a cache and the serial number must | ||
2552 | check out */ | ||
2553 | if (dircache_runinfo.handle && get_file_serialnum(dcfilep)) | ||
2554 | len = get_path_sub(dcfilep->idx, buf, size); | ||
2555 | |||
2556 | if (len < 0) | ||
2557 | errno = ENOENT; | ||
2558 | |||
2559 | dircache_unlock(); | ||
2560 | return len; | ||
1249 | } | 2561 | } |
1250 | 2562 | ||
1251 | static struct dircache_entry* dircache_new_entry(const char *path, int attribute) | 2563 | /** |
1252 | { | 2564 | * searches the sublist starting at 'idx' for the named component |
1253 | struct dircache_entry *entry; | 2565 | */ |
1254 | char basedir[MAX_PATH*2]; | ||
1255 | char *new; | ||
1256 | long last_cache_size = dircache_size; | ||
1257 | 2566 | ||
1258 | strlcpy(basedir, path, sizeof(basedir)); | 2567 | /* helper for get_file_sub() */ |
1259 | new = strrchr(basedir, '/'); | 2568 | static struct dircache_entry * |
1260 | if (new == NULL) | 2569 | get_file_sub_scan(int idx, const char *name, size_t length, int *idxp) |
2570 | { | ||
2571 | struct dircache_entry *ce = get_entry(idx); | ||
2572 | if (ce) | ||
1261 | { | 2573 | { |
1262 | logf("error occurred"); | 2574 | char entname[MAX_NAME+1]; |
1263 | dircache_initialized = false; | 2575 | name = strmemdupa(name, length); |
1264 | return NULL; | 2576 | |
2577 | do | ||
2578 | { | ||
2579 | entry_name_copy(entname, ce); | ||
2580 | if (!strcasecmp(entname, name)) | ||
2581 | { | ||
2582 | *idxp = idx; | ||
2583 | break; | ||
2584 | } | ||
2585 | |||
2586 | idx = ce->next; | ||
2587 | } | ||
2588 | while ((ce = get_entry(idx))); | ||
1265 | } | 2589 | } |
1266 | 2590 | ||
1267 | *new = '\0'; | 2591 | return ce; |
1268 | new++; | 2592 | } |
2593 | |||
2594 | /** | ||
2595 | * searches for the subcomponent of *pathp | ||
2596 | */ | ||
2597 | |||
2598 | /* helper for dircache_get_file() */ | ||
2599 | static int get_file_sub(const char **pathp, int *downp, int *idxp) | ||
2600 | { | ||
2601 | int rc; | ||
2602 | const char *name; | ||
2603 | rc = parse_path_component(pathp, &name, false); | ||
2604 | if (rc <= 0) | ||
2605 | return rc; | ||
2606 | else if (rc >= MAX_PATH) | ||
2607 | return ENAMETOOLONG; /* that's just unpossible, man */ | ||
2608 | |||
2609 | struct dircache_entry *ce = get_file_sub_scan(*downp, name, rc, idxp); | ||
1269 | 2610 | ||
1270 | entry = dircache_get_entry(basedir, true); | 2611 | if (!ce) |
1271 | if (entry == NULL) | 2612 | rc = RC_NOT_FOUND; /* not there; tellibry solly */ |
2613 | else if (!*pathp) | ||
2614 | rc = RC_PATH_ENDED; /* done */ | ||
2615 | else if (!(ce->attr & ATTR_DIRECTORY)) | ||
2616 | rc = ENOTDIR; /* a parent component must be a directory */ | ||
2617 | else | ||
2618 | while ((rc = get_file_sub(pathp, &ce->down, idxp)) == RC_CONTINUE); | ||
2619 | |||
2620 | switch (rc) | ||
1272 | { | 2621 | { |
1273 | logf("basedir not found!"); | 2622 | case RC_GO_UP: /* hit ".."; drop to previous level */ |
1274 | logf("%s", basedir); | 2623 | return RC_CONTINUE; |
1275 | dircache_initialized = false; | 2624 | case RC_PATH_ENDED: /* success! */ |
1276 | return NULL; | 2625 | return RC_FOUND; |
2626 | default: /* component not found or error */ | ||
2627 | return rc; | ||
1277 | } | 2628 | } |
2629 | } | ||
1278 | 2630 | ||
1279 | if (reserve_used + 2*sizeof(struct dircache_entry) + strlen(new)+1 | 2631 | /** |
1280 | >= DIRCACHE_RESERVE) | 2632 | * usermode function to return dircache file info for the given path |
2633 | * | ||
2634 | * returns: | ||
2635 | * success: the volume number that is specified for the file | ||
2636 | * failure: a negative value | ||
2637 | * | ||
2638 | * errors: | ||
2639 | * ENOENT - the file or directory does not exist or path is empty | ||
2640 | * ENAMETOOLONG - a component of the path is too long | ||
2641 | * ENOTDIR - a component of the path is not a directory | ||
2642 | */ | ||
2643 | int dircache_get_file(const char *path, struct dircache_file *dcfilep) | ||
2644 | { | ||
2645 | if (!path_is_absolute(path) || !dcfilep) | ||
1281 | { | 2646 | { |
1282 | logf("not enough space"); | 2647 | errno = ENOENT; |
1283 | dircache_initialized = false; | 2648 | return -1; |
1284 | return NULL; | ||
1285 | } | 2649 | } |
1286 | |||
1287 | while (entry->next != NULL) | ||
1288 | entry = entry->next; | ||
1289 | 2650 | ||
1290 | if (entry->d_name != NULL) | 2651 | dircache_lock(); |
2652 | |||
2653 | if (!dircache_runinfo.handle) | ||
1291 | { | 2654 | { |
1292 | entry = dircache_gen_next(entry); | 2655 | dircache_unlock(); |
1293 | if (entry == NULL) | 2656 | errno = ENOENT; |
2657 | return -2; | ||
2658 | } | ||
2659 | |||
2660 | int volume = 0; | ||
2661 | int idx = 0; | ||
2662 | dc_serial_t serialnum = 0; | ||
2663 | struct dircache_volume *dcvolp = NULL; | ||
2664 | struct dircache_entry *ce = NULL; | ||
2665 | |||
2666 | int rc = RC_GO_UP; | ||
2667 | |||
2668 | while (rc == RC_CONTINUE || rc == RC_GO_UP) | ||
2669 | { | ||
2670 | #ifdef HAVE_MULTIVOLUME | ||
2671 | if (rc == RC_GO_UP) | ||
1294 | { | 2672 | { |
1295 | dircache_initialized = false; | 2673 | volume = path_strip_volume(path, &path, false); |
1296 | return NULL; | 2674 | if (!CHECK_VOL(volume)) |
2675 | { | ||
2676 | rc = ENXIO; | ||
2677 | break; | ||
2678 | } | ||
1297 | } | 2679 | } |
1298 | } | 2680 | #endif /* HAVE_MULTIVOLUME */ |
2681 | |||
2682 | dcvolp = DCVOL(volume); | ||
1299 | 2683 | ||
1300 | size_t size = strlen(new) + 1; | 2684 | int *downp = &dcvolp->root_down; |
1301 | entry->d_name = (d_names_start -= size); | 2685 | if (*downp <= 0) |
1302 | entry->startcluster = 0; | 2686 | { |
1303 | memset(&entry->info, 0, sizeof(entry->info)); | 2687 | rc = ENXIO; |
1304 | entry->info.attribute = attribute; | 2688 | break; |
2689 | } | ||
1305 | 2690 | ||
1306 | strcpy(entry->d_name, new); | 2691 | rc = get_file_sub(&path, downp, &idx); |
1307 | dircache_size += size; | 2692 | } |
1308 | 2693 | ||
1309 | if (attribute & ATTR_DIRECTORY) | 2694 | switch (rc) |
1310 | { | 2695 | { |
1311 | logf("gen_down"); | 2696 | case RC_FOUND: /* hit: component found */ |
1312 | dircache_gen_down(entry); | 2697 | serialnum = ce->serialnum; |
2698 | rc = volume; | ||
2699 | break; | ||
2700 | case RC_PATH_ENDED: /* hit: it's a root (volume or system) */ | ||
2701 | idx = -volume - 1; | ||
2702 | serialnum = dcvolp->serialnum; | ||
2703 | rc = volume; | ||
2704 | break; | ||
2705 | case RC_NOT_FOUND: /* miss */ | ||
2706 | rc = ENOENT; | ||
2707 | default: | ||
2708 | idx = 0; | ||
2709 | errno = rc; | ||
2710 | rc = -3; | ||
2711 | break; | ||
1313 | } | 2712 | } |
1314 | |||
1315 | reserve_used += dircache_size - last_cache_size; | ||
1316 | 2713 | ||
1317 | return entry; | 2714 | dcfilep->idx = idx; |
2715 | dcfilep->serialnum = serialnum; | ||
2716 | |||
2717 | dircache_unlock(); | ||
2718 | return rc; | ||
1318 | } | 2719 | } |
2720 | #endif /* 0 */ | ||
2721 | |||
1319 | 2722 | ||
1320 | void dircache_bind(int fd, const char *path) | 2723 | /** Debug screen/info stuff **/ |
2724 | |||
2725 | /** | ||
2726 | * return cache state parameters | ||
2727 | */ | ||
2728 | void dircache_get_info(struct dircache_info *info) | ||
1321 | { | 2729 | { |
1322 | struct dircache_entry *entry; | 2730 | static const char * const status_descriptions[] = |
1323 | 2731 | { | |
1324 | /* Queue requests until dircache has been built. */ | 2732 | [DIRCACHE_IDLE] = "Idle", |
1325 | if (!dircache_initialized && dircache_is_initializing()) | 2733 | [DIRCACHE_SCANNING] = "Scanning", |
2734 | [DIRCACHE_READY] = "Ready", | ||
2735 | }; | ||
2736 | |||
2737 | if (!info) | ||
2738 | return; | ||
2739 | |||
2740 | memset(info, 0, sizeof (*info)); | ||
2741 | |||
2742 | dircache_lock(); | ||
2743 | |||
2744 | enum dircache_status status = DIRCACHE_IDLE; | ||
2745 | |||
2746 | for (unsigned int volume = 0; volume < NUM_VOLUMES; volume++) | ||
1326 | { | 2747 | { |
1327 | if (fdbind_idx >= MAX_PENDING_BINDINGS) | 2748 | struct dircache_volume *dcvolp = DCVOL(volume); |
1328 | return ; | 2749 | enum dircache_status volstatus = dcvolp->status; |
1329 | strlcpy(fdbind_cache[fdbind_idx].path, path, | 2750 | |
1330 | sizeof(fdbind_cache[fdbind_idx].path)); | 2751 | switch (volstatus) |
1331 | fdbind_cache[fdbind_idx].fd = fd; | 2752 | { |
1332 | fdbind_idx++; | 2753 | case DIRCACHE_SCANNING: |
1333 | return ; | 2754 | /* if any one is scanning then overall status is "scanning" */ |
2755 | status = volstatus; | ||
2756 | |||
2757 | /* sum the time the scanning has taken so far */ | ||
2758 | info->build_ticks += current_tick - dcvolp->start_tick; | ||
2759 | break; | ||
2760 | case DIRCACHE_READY: | ||
2761 | /* if all the rest are idle and at least one is ready, then | ||
2762 | status is "ready". */ | ||
2763 | if (status == DIRCACHE_IDLE) | ||
2764 | status = DIRCACHE_READY; | ||
2765 | |||
2766 | /* sum the build ticks of all "ready" volumes */ | ||
2767 | info->build_ticks += dcvolp->build_ticks; | ||
2768 | break; | ||
2769 | case DIRCACHE_IDLE: | ||
2770 | /* if all are idle; then the whole cache is "idle" */ | ||
2771 | break; | ||
2772 | } | ||
1334 | } | 2773 | } |
1335 | |||
1336 | if (!dircache_initialized) | ||
1337 | return ; | ||
1338 | 2774 | ||
1339 | logf("bind: %d/%s", fd, path); | 2775 | info->status = status; |
1340 | entry = dircache_get_entry(path, false); | 2776 | info->statusdesc = status_descriptions[status]; |
1341 | if (entry == NULL) | 2777 | info->last_size = dircache.last_size; |
2778 | info->size_limit = DIRCACHE_LIMIT; | ||
2779 | info->reserve = DIRCACHE_RESERVE; | ||
2780 | |||
2781 | /* report usage only if there is something ready or being built */ | ||
2782 | if (status != DIRCACHE_IDLE) | ||
1342 | { | 2783 | { |
1343 | logf("not found!"); | 2784 | info->reserve_used = reserve_buf_used(); |
1344 | dircache_initialized = false; | 2785 | info->size = dircache.size; |
1345 | return ; | 2786 | info->sizeused = dircache.sizeused; |
2787 | info->entry_count = dircache.numentries; | ||
1346 | } | 2788 | } |
1347 | 2789 | ||
1348 | fd_bindings[fd] = entry; | 2790 | dircache_unlock(); |
1349 | } | 2791 | } |
1350 | 2792 | ||
1351 | void dircache_update_filesize(int fd, long newsize, long startcluster) | 2793 | #ifdef DIRCACHE_DUMPSTER |
2794 | /** | ||
2795 | * dump RAW binary of buffer and CSV of all valid paths and volumes to disk | ||
2796 | */ | ||
2797 | void dircache_dump(void) | ||
1352 | { | 2798 | { |
1353 | if (!dircache_initialized || fd < 0) | 2799 | /* open both now so they're in the cache */ |
1354 | return ; | 2800 | int fdbin = open(DIRCACHE_DUMPSTER_BIN, O_WRONLY|O_CREAT|O_TRUNC, 0666); |
1355 | 2801 | int fdcsv = open(DIRCACHE_DUMPSTER_CSV, O_WRONLY|O_CREAT|O_TRUNC, 0666); | |
1356 | if (fd_bindings[fd] == NULL) | 2802 | if (fdbin < 0 || fdcsv < 0) |
1357 | { | 2803 | { |
1358 | logf("dircache fd(%d) access error", fd); | 2804 | close(fdbin); |
1359 | dircache_initialized = false; | 2805 | return; |
1360 | return ; | ||
1361 | } | 2806 | } |
1362 | |||
1363 | fd_bindings[fd]->info.size = newsize; | ||
1364 | fd_bindings[fd]->startcluster = startcluster; | ||
1365 | } | ||
1366 | void dircache_update_filetime(int fd) | ||
1367 | { | ||
1368 | #if CONFIG_RTC == 0 | ||
1369 | (void)fd; | ||
1370 | #else | ||
1371 | short year; | ||
1372 | struct tm *now = get_time(); | ||
1373 | if (!dircache_initialized || fd < 0) | ||
1374 | return ; | ||
1375 | |||
1376 | if (fd_bindings[fd] == NULL) | ||
1377 | { | ||
1378 | logf("dircache fd access error"); | ||
1379 | dircache_initialized = false; | ||
1380 | return ; | ||
1381 | } | ||
1382 | year = now->tm_year+1900-1980; | ||
1383 | fd_bindings[fd]->info.wrtdate = (((year)&0x7f)<<9) | | ||
1384 | (((now->tm_mon+1)&0xf)<<5) | | ||
1385 | (((now->tm_mday)&0x1f)); | ||
1386 | fd_bindings[fd]->info.wrttime = (((now->tm_hour)&0x1f)<<11) | | ||
1387 | (((now->tm_min)&0x3f)<<5) | | ||
1388 | (((now->tm_sec/2)&0x1f)); | ||
1389 | #endif | ||
1390 | } | ||
1391 | 2807 | ||
1392 | void dircache_mkdir(const char *path) | 2808 | trigger_cpu_boost(); |
1393 | { /* Test ok. */ | 2809 | dircache_lock(); |
1394 | if (block_until_ready()) | 2810 | |
1395 | return ; | 2811 | if (dircache_runinfo.handle) |
1396 | 2812 | { | |
1397 | 2813 | buffer_lock(); | |
1398 | logf("mkdir: %s", path); | 2814 | |
1399 | dircache_new_entry(path, ATTR_DIRECTORY); | 2815 | /* bin */ |
1400 | } | 2816 | write(fdbin, dircache_runinfo.p + ENTRYSIZE, |
1401 | 2817 | dircache_runinfo.bufsize + 1); | |
1402 | void dircache_rmdir(const char *path) | 2818 | |
1403 | { /* Test ok. */ | 2819 | /* CSV */ |
1404 | struct dircache_entry *entry; | 2820 | fdprintf(fdcsv, "\"Index\",\"Serialnum\"," |
1405 | 2821 | "\"Path\",\"Frontier\"," | |
1406 | if (block_until_ready()) | 2822 | "\"Attribute\",\"File Size\"," |
1407 | return ; | 2823 | "\"Mod Date\",\"Mod Time\"\n"); |
1408 | 2824 | ||
1409 | logf("rmdir: %s", path); | 2825 | FOR_EACH_VOLUME(-1, volume) |
1410 | entry = dircache_get_entry(path, false); | ||
1411 | if (entry == NULL || entry->down == NULL) | ||
1412 | { | ||
1413 | logf("not found or not a directory!"); | ||
1414 | dircache_initialized = false; | ||
1415 | return ; | ||
1416 | } | ||
1417 | |||
1418 | entry->down = NULL; | ||
1419 | entry->d_name = NULL; | ||
1420 | } | ||
1421 | |||
1422 | /* Remove a file from cache */ | ||
1423 | void dircache_remove(const char *name) | ||
1424 | { /* Test ok. */ | ||
1425 | struct dircache_entry *entry; | ||
1426 | |||
1427 | if (block_until_ready()) | ||
1428 | return ; | ||
1429 | |||
1430 | logf("remove: %s", name); | ||
1431 | |||
1432 | entry = dircache_get_entry(name, false); | ||
1433 | |||
1434 | if (entry == NULL) | ||
1435 | { | ||
1436 | logf("not found!"); | ||
1437 | dircache_initialized = false; | ||
1438 | return ; | ||
1439 | } | ||
1440 | |||
1441 | entry->d_name = NULL; | ||
1442 | } | ||
1443 | |||
1444 | void dircache_rename(const char *oldpath, const char *newpath) | ||
1445 | { /* Test ok. */ | ||
1446 | struct dircache_entry *entry, *newentry; | ||
1447 | struct dircache_entry oldentry; | ||
1448 | char absolute_path[MAX_PATH*2]; | ||
1449 | char *p; | ||
1450 | |||
1451 | if (block_until_ready()) | ||
1452 | return ; | ||
1453 | |||
1454 | logf("rename: %s->%s", oldpath, newpath); | ||
1455 | |||
1456 | entry = dircache_get_entry(oldpath, false); | ||
1457 | if (entry == NULL) | ||
1458 | { | ||
1459 | logf("not found!"); | ||
1460 | dircache_initialized = false; | ||
1461 | return ; | ||
1462 | } | ||
1463 | |||
1464 | /* Delete the old entry. */ | ||
1465 | entry->d_name = NULL; | ||
1466 | |||
1467 | /** If we rename the same filename twice in a row, we need to | ||
1468 | * save the data, because the entry will be re-used. */ | ||
1469 | oldentry = *entry; | ||
1470 | |||
1471 | /* Generate the absolute path for destination if necessary. */ | ||
1472 | if (newpath[0] != '/') | ||
1473 | { | ||
1474 | strlcpy(absolute_path, oldpath, sizeof(absolute_path)); | ||
1475 | p = strrchr(absolute_path, '/'); | ||
1476 | if (!p) | ||
1477 | { | 2826 | { |
1478 | logf("Invalid path"); | 2827 | struct dircache_volume *dcvolp = DCVOL(volume); |
1479 | dircache_initialized = false; | 2828 | if (dcvolp->status == DIRCACHE_IDLE) |
1480 | return ; | 2829 | continue; |
2830 | |||
2831 | #ifdef HAVE_MULTIVOLUME | ||
2832 | char name[VOL_MAX_LEN+1]; | ||
2833 | get_volume_name(volume, name); | ||
2834 | #endif | ||
2835 | fdprintf(fdcsv, | ||
2836 | "%d,%lu," | ||
2837 | "\"%c" IF_MV("%s") "\",%u," | ||
2838 | "0x%08X,0," | ||
2839 | "\"\",\"\"\n", | ||
2840 | -volume-1, dcvolp->serialnum, | ||
2841 | PATH_SEPCH, IF_MV(name,) dcvolp->frontier, | ||
2842 | ATTR_DIRECTORY | ATTR_VOLUME); | ||
1481 | } | 2843 | } |
1482 | 2844 | ||
1483 | *p = '\0'; | 2845 | FOR_EACH_CACHE_ENTRY(ce) |
1484 | strlcpy(p, absolute_path, sizeof(absolute_path)-strlen(p)); | 2846 | { |
1485 | newpath = absolute_path; | 2847 | #ifdef DIRCACHE_NATIVE |
1486 | } | 2848 | time_t mtime = fattime_mktime(ce->wrtdate, ce->wrttime); |
1487 | 2849 | #else | |
1488 | newentry = dircache_new_entry(newpath, entry->info.attribute); | 2850 | time_t mtime = ce->mtime; |
1489 | if (newentry == NULL) | 2851 | #endif |
1490 | { | 2852 | struct tm tm; |
1491 | dircache_initialized = false; | 2853 | gmtime_r(&mtime, &tm); |
1492 | return ; | 2854 | |
2855 | char buf[DC_MAX_NAME + 2]; | ||
2856 | *buf = '\0'; | ||
2857 | int idx = get_index(ce); | ||
2858 | get_path_sub(idx, buf, sizeof (buf)); | ||
2859 | |||
2860 | fdprintf(fdcsv, | ||
2861 | "%d,%lu," | ||
2862 | "\"%s\",%u," | ||
2863 | "0x%08X,%lu," | ||
2864 | "%04d/%02d/%02d," | ||
2865 | "%02d:%02d:%02d\n", | ||
2866 | idx, ce->serialnum, | ||
2867 | buf, ce->frontier, | ||
2868 | ce->attr, (ce->attr & ATTR_DIRECTORY) ? | ||
2869 | 0ul : (unsigned long)ce->filesize, | ||
2870 | tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday, | ||
2871 | tm.tm_hour, tm.tm_min, tm.tm_sec); | ||
2872 | } | ||
2873 | |||
2874 | buffer_unlock(); | ||
1493 | } | 2875 | } |
1494 | 2876 | ||
1495 | newentry->down = oldentry.down; | 2877 | dircache_unlock(); |
1496 | newentry->startcluster = oldentry.startcluster; | 2878 | cancel_cpu_boost(); |
1497 | newentry->info.size = oldentry.info.size; | 2879 | |
1498 | newentry->info.wrtdate = oldentry.info.wrtdate; | 2880 | close(fdbin); |
1499 | newentry->info.wrttime = oldentry.info.wrttime; | 2881 | close(fdcsv); |
1500 | } | 2882 | } |
2883 | #endif /* DIRCACHE_DUMPSTER */ | ||
2884 | |||
1501 | 2885 | ||
1502 | void dircache_add_file(const char *path, long startcluster) | 2886 | /** Misc. stuff **/ |
2887 | |||
2888 | /** | ||
2889 | * set the dircache file to initial values | ||
2890 | */ | ||
2891 | void dircache_dcfile_init(struct dircache_file *dcfilep) | ||
1503 | { | 2892 | { |
1504 | struct dircache_entry *entry; | 2893 | dcfilep->idx = 0; |
1505 | 2894 | dcfilep->serialnum = 0; | |
1506 | if (block_until_ready()) | ||
1507 | return ; | ||
1508 | |||
1509 | logf("add file: %s", path); | ||
1510 | entry = dircache_new_entry(path, 0); | ||
1511 | if (entry == NULL) | ||
1512 | return ; | ||
1513 | |||
1514 | entry->startcluster = startcluster; | ||
1515 | } | 2895 | } |
1516 | 2896 | ||
1517 | static bool is_disable_msg_pending(void) | 2897 | #ifdef HAVE_EEPROM_SETTINGS |
2898 | |||
2899 | #ifdef HAVE_HOTSWAP | ||
2900 | /* NOTE: This is hazardous to the filesystem of any sort of removable | ||
2901 | storage unless it may be determined that the filesystem from save | ||
2902 | to load is identical. If it's not possible to do so in a timely | ||
2903 | manner, it's not worth persisting the cache. */ | ||
2904 | #warning "Don't do this; you'll find the consequences unpleasant." | ||
2905 | #endif | ||
2906 | |||
2907 | /* dircache persistence file header magic */ | ||
2908 | #define DIRCACHE_MAGIC 0x00d0c0a1 | ||
2909 | |||
2910 | /* dircache persistence file header */ | ||
2911 | struct dircache_maindata | ||
1518 | { | 2912 | { |
1519 | return check_event_queue(); | 2913 | uint32_t magic; /* DIRCACHE_MAGIC */ |
2914 | struct dircache dircache; /* metadata of the cache! */ | ||
2915 | uint32_t datacrc; /* CRC32 of data */ | ||
2916 | uint32_t hdrcrc; /* CRC32 of header through datacrc */ | ||
2917 | } __attribute__((packed, aligned (4))); | ||
2918 | |||
2919 | /** | ||
2920 | * verify that the clean status is A-ok | ||
2921 | */ | ||
2922 | static bool dircache_is_clean(bool saving) | ||
2923 | { | ||
2924 | if (saving) | ||
2925 | return dircache.dcvol[0].status == DIRCACHE_READY; | ||
2926 | else | ||
2927 | { | ||
2928 | return dircache.dcvol[0].status == DIRCACHE_IDLE && | ||
2929 | !dircache_runinfo.enabled; | ||
2930 | } | ||
1520 | } | 2931 | } |
1521 | 2932 | ||
1522 | DIR_CACHED* opendir_cached(const char* name) | 2933 | /** |
2934 | * function to load the internal cache structure from disk to initialize | ||
2935 | * the dircache really fast with little disk access. | ||
2936 | */ | ||
2937 | int dircache_load(void) | ||
1523 | { | 2938 | { |
1524 | int dd; | 2939 | logf("Loading directory cache"); |
1525 | DIR_CACHED* pdir = opendirs; | 2940 | int fd = open_dircache_file(O_RDONLY); |
2941 | if (fd < 0) | ||
2942 | return -1; | ||
2943 | |||
2944 | int rc = -1; | ||
2945 | |||
2946 | ssize_t size; | ||
2947 | struct dircache_maindata maindata; | ||
2948 | uint32_t crc; | ||
2949 | int handle = 0; | ||
2950 | bool hasbuffer = false; | ||
1526 | 2951 | ||
1527 | if ( name[0] != '/' ) | 2952 | size = sizeof (maindata); |
2953 | if (read(fd, &maindata, size) != size) | ||
1528 | { | 2954 | { |
1529 | DEBUGF("Only absolute paths supported right now\n"); | 2955 | logf("dircache: header read failed"); |
1530 | return NULL; | 2956 | goto error_nolock; |
1531 | } | 2957 | } |
1532 | 2958 | ||
1533 | /* find a free dir descriptor */ | 2959 | /* sanity check the header */ |
1534 | for ( dd=0; dd<MAX_OPEN_DIRS; dd++, pdir++) | 2960 | if (maindata.magic != DIRCACHE_MAGIC) |
1535 | if ( !pdir->busy ) | 2961 | { |
1536 | break; | 2962 | logf("dircache: invalid header magic"); |
2963 | goto error_nolock; | ||
2964 | } | ||
1537 | 2965 | ||
1538 | if ( dd == MAX_OPEN_DIRS ) | 2966 | crc = crc_32(&maindata, offsetof(struct dircache_maindata, hdrcrc), |
2967 | 0xffffffff); | ||
2968 | if (crc != maindata.hdrcrc) | ||
1539 | { | 2969 | { |
1540 | DEBUGF("Too many dirs open\n"); | 2970 | logf("dircache: invalid header CRC32"); |
1541 | errno = EMFILE; | 2971 | goto error_nolock; |
1542 | return NULL; | ||
1543 | } | 2972 | } |
1544 | |||
1545 | pdir->busy = true; | ||
1546 | 2973 | ||
1547 | if (!dircache_initialized || is_disable_msg_pending()) | 2974 | if (maindata.dircache.size != |
2975 | maindata.dircache.sizeentries + maindata.dircache.sizenames || | ||
2976 | ALIGN_DOWN(maindata.dircache.size, ENTRYSIZE) != maindata.dircache.size || | ||
2977 | filesize(fd) - sizeof (maindata) != maindata.dircache.size) | ||
1548 | { | 2978 | { |
1549 | pdir->internal_entry = -1; | 2979 | logf("dircache: file header error"); |
1550 | pdir->regulardir = opendir_uncached(name); | 2980 | goto error_nolock; |
1551 | } | 2981 | } |
1552 | else | 2982 | |
2983 | /* allocate so that exactly the reserve size remains */ | ||
2984 | size_t bufsize = maindata.dircache.size + DIRCACHE_RESERVE + 1; | ||
2985 | handle = alloc_cache(bufsize); | ||
2986 | if (handle <= 0) | ||
2987 | { | ||
2988 | logf("dircache: failed load allocation"); | ||
2989 | goto error_nolock; | ||
2990 | } | ||
2991 | |||
2992 | dircache_lock(); | ||
2993 | buffer_lock(); | ||
2994 | |||
2995 | if (!dircache_is_clean(false)) | ||
2996 | goto error; | ||
2997 | |||
2998 | /* from this point on, we're actually dealing with the cache in RAM */ | ||
2999 | dircache = maindata.dircache; | ||
3000 | |||
3001 | set_buffer(handle, bufsize); | ||
3002 | hasbuffer = true; | ||
3003 | |||
3004 | /* convert back to in-RAM representation */ | ||
3005 | dircache.numentries = maindata.dircache.sizeentries / ENTRYSIZE; | ||
3006 | |||
3007 | /* read the dircache file into memory; start with the entries */ | ||
3008 | size = maindata.dircache.sizeentries; | ||
3009 | if (read(fd, dircache_runinfo.pentry + 1, size) != size) | ||
1553 | { | 3010 | { |
1554 | pdir->regulardir = NULL; | 3011 | logf("dircache read failed #1"); |
1555 | pdir->internal_entry = dircache_get_entry_id_ex(name, true); | 3012 | goto error; |
1556 | pdir->theent.info.attribute = -1; /* used to make readdir_cached aware of the first call */ | ||
1557 | } | 3013 | } |
1558 | 3014 | ||
1559 | if (pdir->internal_entry == -1 && pdir->regulardir == NULL) | 3015 | crc = crc_32(dircache_runinfo.pentry + 1, size, 0xffffffff); |
3016 | |||
3017 | /* continue with the names; fix up indexes to them if needed */ | ||
3018 | dircache.names -= maindata.dircache.sizenames; | ||
3019 | *get_name(dircache.names - 1) = 0; | ||
3020 | |||
3021 | size = maindata.dircache.sizenames; | ||
3022 | if (read(fd, get_name(dircache.names), size) != size) | ||
1560 | { | 3023 | { |
1561 | pdir->busy = false; | 3024 | logf("dircache read failed #2"); |
1562 | return NULL; | 3025 | goto error; |
1563 | } | 3026 | } |
1564 | 3027 | ||
1565 | return pdir; | 3028 | crc = crc_32(get_name(dircache.names), size, crc); |
1566 | } | 3029 | if (crc != maindata.datacrc) |
3030 | { | ||
3031 | logf("dircache: data failed CRC32"); | ||
3032 | goto error; | ||
3033 | } | ||
1567 | 3034 | ||
1568 | struct dirent_cached* readdir_cached(DIR_CACHED* dir) | 3035 | /* only names will be changed in relative position so fix up those |
1569 | { | 3036 | references */ |
1570 | struct dircache_entry *ce = get_entry(dir->internal_entry); | 3037 | ssize_t offset = dircache.names - maindata.dircache.names; |
1571 | struct dirent_uncached *regentry; | 3038 | if (offset != 0) |
1572 | 3039 | { | |
1573 | if (!dir->busy) | 3040 | /* nothing should be open besides the dircache file itself therefore |
1574 | return NULL; | 3041 | no bindings need be resolved; the cache will have its own entry |
3042 | but that should get cleaned up when removing the file */ | ||
3043 | FOR_EACH_CACHE_ENTRY(ce) | ||
3044 | { | ||
3045 | if (!ce->tinyname) | ||
3046 | ce->name += offset; | ||
3047 | } | ||
3048 | } | ||
1575 | 3049 | ||
1576 | if (dir->regulardir != NULL) | 3050 | dircache.reserve_used = 0; |
1577 | { | 3051 | |
1578 | regentry = readdir_uncached(dir->regulardir); | 3052 | /* enable the cache but do not try to build it */ |
1579 | if (regentry == NULL) | 3053 | dircache_enable_internal(false); |
1580 | return NULL; | 3054 | |
1581 | 3055 | /* cache successfully loaded */ | |
1582 | strlcpy(dir->theent.d_name, regentry->d_name, MAX_PATH); | 3056 | logf("Done, %ld KiB used", dircache.size / 1024); |
1583 | dir->theent.startcluster = regentry->startcluster; | 3057 | rc = 0; |
1584 | dir->theent.info = regentry->info; | 3058 | error: |
1585 | 3059 | if (rc < 0 && hasbuffer) | |
1586 | return &dir->theent; | 3060 | reset_buffer(); |
1587 | } | ||
1588 | |||
1589 | /* if theent.attribute=-1 then this is the first call */ | ||
1590 | /* otherwise, this is is not so we first take the entry's ->next */ | ||
1591 | /* NOTE: normal file can't have attribute=-1 */ | ||
1592 | if(dir->theent.info.attribute != -1) | ||
1593 | ce = ce->next; | ||
1594 | /* skip unused entries */ | ||
1595 | while(ce != NULL && ce->d_name == NULL) | ||
1596 | ce = ce->next; | ||
1597 | |||
1598 | if (ce == NULL) | ||
1599 | return NULL; | ||
1600 | 3061 | ||
1601 | strlcpy(dir->theent.d_name, ce->d_name, MAX_PATH); | 3062 | buffer_unlock(); |
1602 | /* Can't do `dir->theent = *ce` | 3063 | dircache_unlock(); |
1603 | because that modifies the d_name pointer. */ | ||
1604 | dir->theent.startcluster = ce->startcluster; | ||
1605 | dir->theent.info = ce->info; | ||
1606 | dir->internal_entry = ce - dircache_root; | ||
1607 | 3064 | ||
1608 | //logf("-> %s", ce->d_name); | 3065 | error_nolock: |
1609 | return &dir->theent; | 3066 | if (rc < 0 && handle > 0) |
3067 | core_free(handle); | ||
3068 | |||
3069 | if (fd >= 0) | ||
3070 | close(fd); | ||
3071 | |||
3072 | remove_dircache_file(); | ||
3073 | return rc; | ||
1610 | } | 3074 | } |
1611 | 3075 | ||
1612 | int closedir_cached(DIR_CACHED* dir) | 3076 | /** |
3077 | * function to save the internal cache stucture to disk for fast loading | ||
3078 | * on boot | ||
3079 | */ | ||
3080 | int dircache_save(void) | ||
1613 | { | 3081 | { |
1614 | if (!dir->busy) | 3082 | logf("Saving directory cache"); |
3083 | |||
3084 | int fd = open_dircache_file(O_WRONLY|O_CREAT|O_TRUNC|O_APPEND); | ||
3085 | if (fd < 0) | ||
1615 | return -1; | 3086 | return -1; |
1616 | |||
1617 | dir->busy=false; | ||
1618 | if (dir->regulardir != NULL) | ||
1619 | return closedir_uncached(dir->regulardir); | ||
1620 | |||
1621 | return 0; | ||
1622 | } | ||
1623 | 3087 | ||
1624 | int mkdir_cached(const char *name) | 3088 | dircache_lock(); |
1625 | { | 3089 | buffer_lock(); |
1626 | int rc=mkdir_uncached(name); | 3090 | |
1627 | if (rc >= 0) | 3091 | int rc = -1; |
1628 | dircache_mkdir(name); | 3092 | |
1629 | return(rc); | 3093 | if (!dircache_is_clean(true)) |
3094 | goto error; | ||
3095 | |||
3096 | /* save the header structure along with the cache metadata */ | ||
3097 | ssize_t size; | ||
3098 | uint32_t crc; | ||
3099 | struct dircache_maindata maindata = | ||
3100 | { | ||
3101 | .magic = DIRCACHE_MAGIC, | ||
3102 | .dircache = dircache, | ||
3103 | }; | ||
3104 | |||
3105 | /* store the size since it better detects an invalid header */ | ||
3106 | maindata.dircache.sizeentries = maindata.dircache.numentries * ENTRYSIZE; | ||
3107 | |||
3108 | /* write the template header */ | ||
3109 | size = sizeof (maindata); | ||
3110 | if (write(fd, &maindata, size) != size) | ||
3111 | { | ||
3112 | logf("dircache: write failed #1"); | ||
3113 | goto error; | ||
3114 | } | ||
3115 | |||
3116 | /* write the dircache entries */ | ||
3117 | size = maindata.dircache.sizeentries; | ||
3118 | if (write(fd, dircache_runinfo.pentry + 1, size) != size) | ||
3119 | { | ||
3120 | logf("dircache: write failed #2"); | ||
3121 | goto error; | ||
3122 | } | ||
3123 | |||
3124 | crc = crc_32(dircache_runinfo.pentry + 1, size, 0xffffffff); | ||
3125 | |||
3126 | /* continue with the names */ | ||
3127 | size = maindata.dircache.sizenames; | ||
3128 | if (write(fd, get_name(dircache.names), size) != size) | ||
3129 | { | ||
3130 | logf("dircache: write failed #3"); | ||
3131 | goto error; | ||
3132 | } | ||
3133 | |||
3134 | crc = crc_32(get_name(dircache.names), size, crc); | ||
3135 | maindata.datacrc = crc; | ||
3136 | |||
3137 | /* rewrite the header with CRC info */ | ||
3138 | if (lseek(fd, 0, SEEK_SET) != 0) | ||
3139 | { | ||
3140 | logf("dircache: seek failed"); | ||
3141 | goto error; | ||
3142 | } | ||
3143 | |||
3144 | crc = crc_32(&maindata, offsetof(struct dircache_maindata, hdrcrc), | ||
3145 | 0xffffffff); | ||
3146 | maindata.hdrcrc = crc; | ||
3147 | |||
3148 | if (write(fd, &maindata, sizeof (maindata)) != sizeof (maindata)) | ||
3149 | { | ||
3150 | logf("dircache: write failed #4"); | ||
3151 | goto error; | ||
3152 | } | ||
3153 | |||
3154 | /* as of now, no changes to the volumes should be allowed at all since | ||
3155 | that makes what was saved completely invalid */ | ||
3156 | rc = 0; | ||
3157 | error: | ||
3158 | buffer_unlock(); | ||
3159 | dircache_unlock(); | ||
3160 | |||
3161 | if (rc < 0) | ||
3162 | remove_dircache_file(); | ||
3163 | |||
3164 | close(fd); | ||
3165 | return rc; | ||
1630 | } | 3166 | } |
3167 | #endif /* HAVE_EEPROM_SETTINGS */ | ||
1631 | 3168 | ||
1632 | int rmdir_cached(const char* name) | 3169 | /** |
3170 | * main one-time initialization function that must be called before any other | ||
3171 | * operations within the dircache | ||
3172 | */ | ||
3173 | void dircache_init(size_t last_size) | ||
1633 | { | 3174 | { |
1634 | int rc=rmdir_uncached(name); | 3175 | queue_init(&dircache_queue, false); |
1635 | if(rc >= 0) | 3176 | |
1636 | dircache_rmdir(name); | 3177 | dircache.last_size = MIN(last_size, DIRCACHE_LIMIT); |
1637 | return(rc); | 3178 | |
3179 | struct dircache_runinfo *dcrip = &dircache_runinfo; | ||
3180 | dcrip->suspended = 1; | ||
3181 | dcrip->thread_done = true; | ||
3182 | dcrip->ops.move_callback = move_callback; | ||
1638 | } | 3183 | } |