aboutsummaryrefslogtreecommitdiff
path: root/fs/squashfs
Commit message (Collapse)AuthorAgeFilesLines
* ANDROID: squashfs: Fix endianness issueDaniel Rosenberg2017-06-281-5/+6
| | | | | | | | | | Code in squashfs_process_blocks was not correctly assigning length. Casting to u16* introduced endianness issues on some architectures. Signed-off-by: Daniel Rosenberg <drosen@google.com> Bug: 35257858 Change-Id: I9efaef4bc531b7469de79cf94738ade2dd6e6a8c
* ANDROID: squashfs: Fix signed division issueDaniel Rosenberg2017-06-281-2/+2
| | | | | | | | | | The value here can change depending on the type that PAGE_SIZE has on a given architecture. To avoid the ensuing signed and unsigned division conversions, we shift instead using PAGE_SHIFT Signed-off-by: Daniel Rosenberg <drosen@google.com> Bug: 35257858 Change-Id: I132cae93abea39390c3f0f91a4b2e026e97ed4c7
* ANDROID: Squashfs: fix missing NULL checkDaniel Rosenberg2017-04-111-3/+5
| | | | | | | | | "Squashfs: refactor page_actor" missed a null check within squashfs_bh_to_actor, as that array may now contain null entries Signed-off-by: Daniel Rosenberg <drosen@google.com> Bug: 35257858 Change-Id: Ia6748af980859237a96f509f35903f722a46fd59
* fs/squashfs: change usage of LZ4 to work with new LZ4 versionSven Schmidt2017-04-111-6/+6
| | | | | | | | | | | | | | | | | | | | Update fs/squashfs to use the updated functions from the new LZ4 module. Link: http://lkml.kernel.org/r/1486321748-19085-5-git-send-email-4sschmid@informatik.uni-hamburg.de Signed-off-by: Sven Schmidt <4sschmid@informatik.uni-hamburg.de> Cc: Bongkyu Kim <bongkyu.kim@lge.com> Cc: Rui Salvaterra <rsalvaterra@gmail.com> Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: David S. Miller <davem@davemloft.net> Cc: Anton Vorontsov <anton@enomsg.org> Cc: Colin Cross <ccross@android.com> Cc: Kees Cook <keescook@chromium.org> Cc: Tony Luck <tony.luck@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* Squashfs: optimize reading uncompressed dataAdrien Schildknecht2017-04-112-7/+55
| | | | | | | | | | | | | | | | | | | When dealing with uncompressed data, there is no need to read a whole block (default 128K) to get the desired page: the pages are independent from each others. This patch change the readpages logic so that reading uncompressed data only read the number of pages advised by the readahead algorithm. Moreover, if the page actor contains holes (i.e. pages that are already up-to-date), squashfs skips the buffer_head associated to those pages. This patch greatly improve the performance of random reads for uncompressed files because squashfs only read what is needed. It also reduces the number of unnecessary reads. Signed-off-by: Adrien Schildknecht <adriens@google.com> Change-Id: I9109e431ac180a4e0c65381a3e754a425672d3c0
* Squashfs: implement .readpages()Adrien Schildknecht2017-04-113-57/+150
| | | | | | | | | | | | | | Squashfs does not implement .readpages(), so the kernel just repeatedly calls .readpage(). The readpages function tries to pack as much pages as possible in the same page actor so that only 1 read request is issued. Now that the read requests are asynchronous, the kernel can truly prefetch pages using its readahead algorithm. Signed-off-by: Adrien Schildknecht <adriens@google.com> Change-Id: I09590c56876e931acca9070505476a98ab1ca79e
* Squashfs: replace buffer_head with BIOAdrien Schildknecht2017-04-114-272/+455
| | | | | | | | | | | | | | | | | | The 'll_rw_block' has been deprecated and BIO is now the basic container for block I/O within the kernel. Switching to BIO offers 2 advantages: 1/ It removes synchronous wait for the up-to-date buffers: SquashFS now deals with decompressions/copies asynchronously. Implementing an asynchronous mechanism to read data is needed to efficiently implement .readpages(). 2/ Prior to this patch, merging the read requests entirely depends on the IO scheduler. SquashFS has more information than the IO scheduler about what could be merged. Moreover, merging the reads at the FS level means that we rely less on the IO scheduler. Signed-off-by: Adrien Schildknecht <adriens@google.com> Change-Id: I98cf130f1977d396cdeb1f0617425e208b7d0214
* Squashfs: refactor page_actorAdrien Schildknecht2017-04-1110-213/+255
| | | | | | | | | | | | | | | | | This patch essentially does 3 things: 1/ Always use an array of page to store the data instead of a mix of buffers and pages. 2/ It is now possible to have 'holes' in a page actor, i.e. NULL pages in the array. When reading a block (default 128K), squashfs tries to grab all the pages covering this block. If a single page is up-to-date or locked, it falls back to using an intermediate buffer to do the read and then copy the pages in the actor. Allowing holes in the page actor remove the need for this intermediate buffer. 3/ Refactor the wrappers to share code that deals with page actors. Signed-off-by: Adrien Schildknecht <adriens@google.com> Change-Id: I73f736611766962857c423320478af8cfab8df38
* Squashfs: remove the FILE_CACHE optionAdrien Schildknecht2017-04-114-109/+2
| | | | | | | | | | | FILE_DIRECT is working fine and offers faster results and lower memory footprint. Removing FILE_CACHE makes our life easier because we don't have to maintain 2 differents function that does the same thing. Signed-off-by: Adrien Schildknecht <adriens@google.com> Change-Id: I3e68dda86fc5d6e67b7e31635974201948e3c4a7
* Squashfs: Add LZ4 compression configuration optionPhillip Lougher2017-04-114-0/+27
| | | | | | | Add the glue code, and also update the documentation. Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Change-Id: I47b7b1777547d48f6924e6bd2be3d865b438b617
* Squashfs: add LZ4 compression supportPhillip Lougher2017-04-112-0/+143
| | | | | | | | | | Add support for reading file systems compressed with the LZ4 compression algorithm. This patch adds the LZ4 decompressor wrapper code. Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Change-Id: Id3571428aa71615774f8e542e76ade33a0288564
* fs/squashfs/super.c: logging cleanupFabian Frederick2017-04-111-2/+3
| | | | | | | | | | | | - Convert printk to pr_foo() - Add pr_fmt for future logging entries - Coalesce formats Signed-off-by: Fabian Frederick <fabf@skynet.be> Cc: Phillip Lougher <phillip@squashfs.org.uk> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Change-Id: I4faa93f10f9a28beed3023d0f99404a1b6a4ca8f
* fs/squashfs/file_direct.c: replace count*size kmalloc by kmalloc_arrayFabian Frederick2017-04-111-1/+1
| | | | | | | | | | | kmalloc_array() manages count*sizeof overflow. Signed-off-by: Fabian Frederick <fabf@skynet.be> Cc: Phillip Lougher <phillip@squashfs.org.uk> Cc: Joe Perches <joe@perches.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Change-Id: I625d688316193f1e1f7d32e715c9fecb57b58356
* fs/squashfs/squashfs.h: replace pr_warning by pr_warnFabian Frederick2017-04-111-1/+1
| | | | | | | | | | Update the last pr_warning callsite in fs branch Signed-off-by: Fabian Frederick <fabf@skynet.be> Cc: Phillip Lougher <phillip@squashfs.org.uk> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Change-Id: Ied766768f7b03e554bb31fab2a55cbb7286f5c8f
* Squashfs: fix failure to unlock pages on decompress errorPhillip Lougher2017-04-111-1/+4
| | | | | | | | | | | Direct decompression into the page cache. If we fall back to using an intermediate buffer (because we cannot grab all the page cache pages) and we get a decompress fail, we forgot to release the pages. Reported-by: Roman Peniaev <r.peniaev@gmail.com> Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Change-Id: I2c7b391bd5f99a9c8af55222d28db41742768335
* Squashfs: Check stream is not NULL in decompressor_multi.cPhillip Lougher2017-04-111-4/+3
| | | | | | | | | | Fix static checker complaint that stream is not checked in squashfs_decompressor_destroy(). Reported-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Reviewed-by: Minchan Kim <minchan@kernel.org> Change-Id: I673b0f5f560c525fb7133c01af4d34e5ed359e36
* Squashfs: Directly decompress into the page cache for file dataPhillip Lougher2017-04-115-1/+336
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This introduces an implementation of squashfs_readpage_block() that directly decompresses into the page cache. This uses the previously added page handler abstraction to push down the necessary kmap_atomic/kunmap_atomic operations on the page cache buffers into the decompressors. This enables direct copying into the page cache without using the slow kmap/kunmap calls. The code detects when multiple threads are racing in squashfs_readpage() to decompress the same block, and avoids this regression by falling back to using an intermediate buffer. This patch enhances the performance of Squashfs significantly when multiple processes are accessing the filesystem simultaneously because it not only reduces memcopying, but it more importantly eliminates the lock contention on the intermediate buffer. Using single-thread decompression. dd if=file1 of=/dev/null bs=4096 & dd if=file2 of=/dev/null bs=4096 & dd if=file3 of=/dev/null bs=4096 & dd if=file4 of=/dev/null bs=4096 Before: 629145600 bytes (629 MB) copied, 45.8046 s, 13.7 MB/s After: 629145600 bytes (629 MB) copied, 9.29414 s, 67.7 MB/s Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Reviewed-by: Minchan Kim <minchan@kernel.org> Change-Id: I6e53be0b49ef135d633e08effdb51e7354253cf7
* Squashfs: Restructure squashfs_readpage()Phillip Lougher2017-04-114-71/+118
| | | | | | | | | | | | | | Restructure squashfs_readpage() splitting it into separate functions for datablocks, fragments and sparse blocks. Move the memcpying (from squashfs cache entry) implementation of squashfs_readpage_block into file_cache.c This allows different implementations to be supported. Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Reviewed-by: Minchan Kim <minchan@kernel.org> Change-Id: Ic915501b91711e56ea3231ba0bf6a6e1429dfd2a
* Squashfs: Generalise paging handling in the decompressorsPhillip Lougher2017-04-1113-67/+163
| | | | | | | | | | | | | | | | | | | | | | Further generalise the decompressors by adding a page handler abstraction. This adds helpers to allow the decompressors to access and process the output buffers in an implementation independant manner. This allows different types of output buffer to be passed to the decompressors, with the implementation specific aspects handled at decompression time, but without the knowledge being held in the decompressor wrapper code. This will allow the decompressors to handle Squashfs cache buffers, and page cache pages. This patch adds the abstraction and an implementation for the caches. Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Reviewed-by: Minchan Kim <minchan@kernel.org> Change-Id: I5f9bcb7e60de194d20418993def49e4b981dae99
* Squashfs: add multi-threaded decompression using percpu variablePhillip Lougher2017-04-113-20/+145
| | | | | | | | | | | | | | | | | | Add a multi-threaded decompression implementation which uses percpu variables. Using percpu variables has advantages and disadvantages over implementations which do not use percpu variables. Advantages: * the nature of percpu variables ensures decompression is load-balanced across the multiple cores. * simplicity. Disadvantages: it limits decompression to one thread per core. Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Change-Id: I11572e11c27ed2224243e4291a5bfe43fbdacd8e
* squashfs: Enhance parallel I/OMinchan Kim2017-04-113-1/+221
| | | | | | | | | | | | | | | | | | | | | | | Now squashfs have used for only one stream buffer for decompression so it hurts parallel read performance so this patch supports multiple decompressor to enhance performance parallel I/O. Four 1G file dd read on KVM machine which has 2 CPU and 4G memory. dd if=test/test1.dat of=/dev/null & dd if=test/test2.dat of=/dev/null & dd if=test/test3.dat of=/dev/null & dd if=test/test4.dat of=/dev/null & old : 1m39s -> new : 9s * From v1 * Change comp_strm with decomp_strm - Phillip * Change/add comments - Phillip Signed-off-by: Minchan Kim <minchan@kernel.org> Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Change-Id: I8d4b33a797ed45e6ce792a58e6c107d6c8eac9ed
* Squashfs: Refactor decompressor interface and codePhillip Lougher2017-04-1111-136/+216
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The decompressor interface and code was written from the point of view of single-threaded operation. In doing so it mixed a lot of single-threaded implementation specific aspects into the decompressor code and elsewhere which makes it difficult to seamlessly support multiple different decompressor implementations. This patch does the following: 1. It removes compressor_options parsing from the decompressor init() function. This allows the decompressor init() function to be dynamically called to instantiate multiple decompressors, without the compressor options needing to be read and parsed each time. 2. It moves threading and all sleeping operations out of the decompressors. In doing so, it makes the decompressors non-blocking wrappers which only deal with interfacing with the decompressor implementation. 3. It splits decompressor.[ch] into decompressor generic functions in decompressor.[ch], and moves the single threaded decompressor implementation into decompressor_single.c. The result of this patch is Squashfs should now be able to support multiple decompressors by adding new decompressor_xxx.c files with specialised implementations of the functions in decompressor_single.c Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Reviewed-by: Minchan Kim <minchan@kernel.org> Change-Id: I79eafe0ea0f9f79f90a94efef083cff452f731e5
* Squashfs: add corruption check for type in squashfs_readdir()Phillip Lougher2017-04-112-3/+9
| | | | | | | | | We read the type field from disk. This value should be sanity checked for correctness to avoid an out of bounds access when reading the squashfs_filetype_table array. Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Change-Id: I0df14ce631a38af3f5920f604e727e88812dcffc
* Squashfs: add corruption check in get_dir_index_using_offset()Phillip Lougher2017-04-111-1/+8
| | | | | | | | | | | | | We read the size (of the name) field from disk. This value should be sanity checked for correctness to avoid blindly reading huge amounts of unnecessary data from disk on corruption. Note, here we're not actually reading the name into a buffer, but skipping it, and so corruption doesn't cause buffer overflow, merely lots of unnecessary amounts of data to be read. Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Change-Id: Ie5c833653d72bb0a89623bfc83e724a5015ea90e
* Squashfs: fix corruption checks in squashfs_readdir()Phillip Lougher2017-04-111-3/+2
| | | | | | | | | | | The dir_count and size fields when read from disk are sanity checked for correctness. However, the sanity checks only check the values are not greater than expected. As dir_count and size were incorrectly defined as signed ints, this can lead to corrupted values appearing as negative which are not trapped. Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Change-Id: Id72c1e19289eb445a29d0a504677c62f18ac1bd2
* Squashfs: fix corruption checks in squashfs_lookup()Phillip Lougher2017-04-111-1/+2
| | | | | | | | | | | The dir_count and size fields when read from disk are sanity checked for correctness. However, the sanity checks only check the values are not greater than expected. As dir_count and size were incorrectly defined as signed ints, this can lead to corrupted values appearing as negative which are not trapped. Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Change-Id: Iadcc635658dcc7dd1ce940cb0537e5ebccca471a
* Squashfs: fix corruption check in get_dir_index_using_name()Phillip Lougher2017-04-111-3/+1
| | | | | | | | | | | | | | | | | Patch "Squashfs: sanity check information from disk" from Dan Carpenter adds a missing check for corruption in the "size" field while reading the directory index from disk. It, however, sets err to -EINVAL, this value is not used later, and so setting it is completely redundant. So remove it. Errors in reading the index are deliberately non-fatal. If we get an error in reading the index we just return the part of the index we have managed to read - the index isn't essential, just quicker. Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Change-Id: I846f6cccb98ff0f5d576983214ff964507f4376a
* Squashfs: Optimized uncompressed buffer loopManish Sharma2017-04-111-7/+4
| | | | | | | | | Merged the two for loops. We might get a little gain by overlapping wait_on_bh and the memcpy operations. Signed-off-by: Manish Sharma <manishrma@gmail.com> Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Change-Id: I7cf983af7e72452b313fdb910eb99bcdc8398ca9
* Squashfs: sanity check information from diskDan Carpenter2017-04-111-1/+6
| | | | | | | | | We read the size of the name from the disk, but a larger name than expected would cause memory corruption. Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk> Change-Id: Ia307b9c023bf1c7c024bb1d90302aea527b66997
* [readdir] convert squashfsAl Viro2017-04-111-28/+12
| | | | | Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> Change-Id: I7afbbb3c74557bc8ab65fc9234952502c354b8e1
* fs: push sync_filesystem() down to the file system's remount_fs()Theodore Ts'o2016-11-071-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, the no-op "mount -o mount /dev/xxx" operation when the file system is already mounted read-write causes an implied, unconditional syncfs(). This seems pretty stupid, and it's certainly documented or guaraunteed to do this, nor is it particularly useful, except in the case where the file system was mounted rw and is getting remounted read-only. However, it's possible that there might be some file systems that are actually depending on this behavior. In most file systems, it's probably fine to only call sync_filesystem() when transitioning from read-write to read-only, and there are some file systems where this is not needed at all (for example, for a pseudo-filesystem or something like romfs). Signed-off-by: "Theodore Ts'o" <tytso@mit.edu> Cc: linux-fsdevel@vger.kernel.org Cc: Christoph Hellwig <hch@infradead.org> Cc: Artem Bityutskiy <dedekind1@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Evgeniy Dushistov <dushistov@mail.ru> Cc: Jan Kara <jack@suse.cz> Cc: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> Cc: Anders Larsen <al@alarsen.net> Cc: Phillip Lougher <phillip@squashfs.org.uk> Cc: Kees Cook <keescook@chromium.org> Cc: Mikulas Patocka <mikulas@artax.karlin.mff.cuni.cz> Cc: Petr Vandrovec <petr@vandrovec.name> Cc: xfs@oss.sgi.com Cc: linux-btrfs@vger.kernel.org Cc: linux-cifs@vger.kernel.org Cc: samba-technical@lists.samba.org Cc: codalist@coda.cs.cmu.edu Cc: linux-ext4@vger.kernel.org Cc: linux-f2fs-devel@lists.sourceforge.net Cc: fuse-devel@lists.sourceforge.net Cc: cluster-devel@redhat.com Cc: linux-mtd@lists.infradead.org Cc: jfs-discussion@lists.sourceforge.net Cc: linux-nfs@vger.kernel.org Cc: linux-nilfs@vger.kernel.org Cc: linux-ntfs-dev@lists.sourceforge.net Cc: ocfs2-devel@oss.oracle.com Cc: reiserfs-devel@vger.kernel.org
* first commitMeizu OpenSource2016-08-1525-0/+4998
generated by cgit v1.2.3 (git 2.39.1) at 2026-03-14 05:42:02 +0000