Scala uniform, fluent access to files, urls and other resources API. Fluent for java too.
- integrate apache commons vfs via Locations.vfs("...")
- UrlLocation transparently manages
- handle redirects
- good RequestHeader defaults
- reuse already opened HttpConnections
- use proxy if using scalaj/http library
- manage backpresure from server
- [TODO] connection timeout temporarly
- [TODO] maximum connections/ip-hostname? for a specified timeframe?
- [TODO] delay between requests to same server
- [TODO] non-blocking IO returning Future/Observable ?
- Version and Etag that is changed if file is changed. A change version/tag doesn't warranty a change in file content.
- Types of Locations
- Abstract Locations
- InputLocation - locations that can be read
- OutputLocation - locations that can be written
- InOutLocation - location that can be read/written
- RelativeLocation - part of a location. Cannot be resolved to some content.
- NavigableLocation - location for which you can find parent/childrens/descendants.
- VersionedLocation - location trait with Version/Etag/UniqueId
- Physical Locations
- FileLocation - HasContent, NoChildren
- FolderLocation - NoContent, HasChildren
- MemoryLocation - read/write in memory content - useful for tests.
- ClasspathLocation - InputLocation from classpath.
- Stream Location - location from a InputStream. Might not be reopened.
- TempLocation - location in the temporary file system.
- UrlLocation - location from a url. Follows redirects if needed
- VfsLocation - location based on Apache-Vfs library
- ZipInputLocation - location around zip files
- Abstract Locations
- natural sorting for listing files
- Operation Options
- CopyOptions: copy metadata forced or optionally if exists
- OperationMonitor: log warnings
//Reading from a file: Locations.file("/home/costin/myfile.txt").readContent //Copying a file to a new folder (and create parent folder if needed) Locations.file("/home/costin/myfile.txt"). copyTo(Locations.file("/home/costin/folder2/myfile2.txt").mkdirOnParentIfNecessary)) //Copying a file to a new relative folder (and create parent folder if needed) Locations.file("/home/costin/myfile.txt"). copyTo(Locations.relative("folder3/myfile2.txt"))) //read content from classpath val text = Locations.classpath("META-INF/maven/org.slf4j/slf4j-api/pom.properties"). readContentAsText.get //get a stream from classpath val text = Locations.classpath("META-INF/maven/org.slf4j/slf4j-api/pom.properties").toInputStream
For more samples see LocationsTest.scala
- from sbt
libraryDependencies += "org.raisercostin" %% "jedi-io" % "0.18"
- maven resolver at bintray - http://dl.bintray.com/raisercostin/maven
resolvers += "raisercostin repository" at "http://dl.bintray.com/raisercostin/maven"
Projects that are using jedi-io:
- https://github.com/raisercostin/my-scala-scripts (see here a script for bulk uploading/importing maven artefacts from a repository/svn/folder to bintray)
- to configure release
- to release
sbt> release skip-tests
- add Haddop/Spark like executor for windows: \
- explain concepts
- add File/Folder concepts
- operations on files/folders
- operations on lists of files/folders (like manual selections or filters in OFM)
- make it async
- make a small 2panel file manager - see trolCommander - other file managers
- Locations.url("file://...") should create a FileLocation?
- add FileStore
- AddHttpsWritable via vfs - http://detailfocused.blogspot.ro/2009/06/add-plugin-for-apache-vfs.html
- add Locations:
- StreamProviderLocation - location that knows how to open a stream
- CachedLocation - location that reads the original location only if the cache expired. Useful around UrlLocation, ZipLocation, etc. Check version if changed read origin.
- MetadataLocation - location that saves metadata associated with the file. See osx files, ds-
- UrlLocation use RequestHeader for mime type and reader encoding. Maybe we need MetadataLocation
- clarify operations
- do actions (need to resolve to the filesystems)
- just change in memory representation
- both Resolved/Absolute and Relative could act as destination if they are resolved with src.
- see from here https://wiki.apache.org/commons/VfsNext . Is already integrated with vfs.
- add transactional aspect as an option http://wiki.c2.com/?TransactionalFileSystem
- store metadata as YAML (hierarchy, use anchors). Should be useful for id3(mp3), exif(image files) etc.
- add hierachical map (apache collections - MultiValueMap & HierarchicalConfiguration, guava - Multimap, spring - MultiValueMap, me - HierarchicalMultimap
- apache commons config
- see config libraries: http://javaeeconfig.blogspot.ro/2014/08/overview-of-existing-configuration.html
- apache Configuration vs typesafe Config
- guava ListMultimap via Serializable - https://stackoverflow.com/questions/17697974/writing-and-reading-listmultimapobject-object-to-file-using-properties
- apache commons config
- FileSystems: sink, source, traverse(list, ...), mount/unmount
- src/from - dest/to
- selected items
- Operations between Items with the Operation Config/Operation Strategy
- including metadata
- using symlinks
A container of keys in form a.b.c is needed. The value could be multivalue eventually typed : Seq(value1,value2,value3). Given a container and a key prefix another container should be returned with partial keys prefix removed. A refereence to full key might be useful. A relativeKey concept might be useful? A save/load from hocon, yaml would be nice. A business wrapper around a Config should be easy to use.