gurl

general URL class, with caching.

class URL(urlpath.URL, urllib.parse._NetlocResultMixinStr, pathlib.PurePath)
 URL(*args, **kwargs)
 
 Derived from 
 https://raw.githubusercontent.com/chrono-meter/urlpath/master/urlpath.py
 
 to provide more compatibility with pathlib.Path functionality
 
 If a request is made to access the URL read_bytes() or read_text()
 and self.cache is True, then self.local_file is used as a cache
 
 Keywords:
   
 suffix_value
 ------------
 If suffix_value is set to a string (e.g. ".store") then this 
 is appended to the cached filename. This can be useful for
 distinguishing cached files from other types.
 
 suffix_value=None : If suffix_value is set to a string (e.g. ".store")
                     then this is appended to the cached filename. 
                     This can be useful for distinguishing cached files 
                     from other types.
 verbose=True      : verbose switch
 log=None          : set to string to send verboise output to file
 pwr=False         : password required (pwr). Set to True is the URL
                     requires username and password. You can then either
                     set 
 
                         self.with_userinfo(username, password)
 
                     explicitly, or be prompted and store in local
                     cylog file (encrypted). The use of cylog is
                     the default if a password is required. Note that
                     if a call to get() without using a password fails,
                     then the code will next try it with a password i.e.
                     try to guess that you meant pwr=True. You can stop
                     that behaviour by setting pwr=None. 
 binary=False      : Set to true if you want to retrieve a binary file.
 cache=False       : Set to True if you want to cache any retrieved 
                     results. The cached filename is self.local_file
 local_dir="."     : Set the name of the root directory for cached
                     files. The default is ".".
                     self.local_file is derived from this, being the
                     local_dir followed by the URL filename path.
 
 Method resolution order:
     URL
     urlpath.URL
     urllib.parse._NetlocResultMixinStr
     urllib.parse._NetlocResultMixinBase
     urllib.parse._ResultMixinStr
     pathlib.PurePath
     builtins.object
 Methods defined here:
 
 __del__(self)
 
 __init__(self, *args, **kwargs)
     initialisation
 
 clear(self)
     clear cached file
 
 get_data(self, data)
     get data from URL
     
     Arguments:
       data : set to not None to skip
 
 get_data_with_login(self, head=False)
     get data from URL with login. Wew try several strategies
     with the login that should allow for redirection
     and 'awkward' logins.
 
 get_data_without_login(self)
     get data from URL without using login
 
 init(self, *args, **kwargs)
     initialisation to update local info
 
 login(self)
 
 msg(self, *args, stderr=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>)
     messaging
 
 read(self)
     Open the URL read it, and close the file.
 
 read_bytes(self)
     Open the URL in bytes mode, read it, and close the file.
 
 read_text(self)
     Open the URL in text mode, read it, and close the file.
 
 stat(self)
     Return the result of the stat() system call on the local
     file for this path, like os.stat() does.
 
 update(self)
 
 write(self, data, local=True)
     Write data to local file if local=True
 
 write_bytes(self, data)
     Open the URL as local file in bytes mode, write it, and close the file.
 
 write_text(self, data)
     Open the URL as local file in text mode, write it, and close the file.
 
 ----------------------------------------------------------------------
 Static methods defined here:
 
 __new__(cls, *args, **kwargs)
     make a new URL, without trailing '/'
     or return Path() if you refer to a local file
 
 ----------------------------------------------------------------------
 Methods inherited from urlpath.URL:
 
 __bytes__(self)
     Return the bytes representation of the path.  This is only
     recommended to use under Unix.
 
 __str__(self)
     Return str(self).
 
 add_query(self, query=None, **kwargs)
     Return a new url with the query ammended.
 
 as_uri(self)
     Return URI.
 
 delete(self, **kwargs)
     Sends a DELETE request.
     
     :param \*\*kwargs: Optional arguments that ``request`` takes.
     :return: :class:`Response <Response>` object
     :rtype: requests.Response
 
 get(self, params=None, **kwargs)
     Sends a GET request.
     
     :param params: (optional) Dictionary or bytes to be sent in the query string for the :class:`Request`.
     :param \*\*kwargs: Optional arguments that ``request`` takes.
     :return: :class:`Response <Response>` object
     :rtype: requests.Response
 
 get_json(self, name='', query='', keys='', overwrite=False)
     Runs a url with a specific query, amending query if necessary, and returns the result after applying a
     transformer
 
 get_text(self, name='', query='', pattern='', overwrite=False)
     Runs a url with a specific query, amending query if necessary, and returns the resulting text
 
 head(self, **kwargs)
     Sends a HEAD request.
     
     :param \*\*kwargs: Optional arguments that ``request`` takes.
     :return: :class:`Response <Response>` object
     :rtype: requests.Response
 
 options(self, **kwargs)
     Sends a OPTIONS request.
     
     :param \*\*kwargs: Optional arguments that ``request`` takes.
     :return: :class:`Response <Response>` object
     :rtype: requests.Response
 
 patch(self, data=None, **kwargs)
     Sends a PATCH request.
     
     :param data: (optional) Dictionary, bytes, or file-like object to send in the body of the :class:`Request`.
     :param \*\*kwargs: Optional arguments that ``request`` takes.
     :return: :class:`Response <Response>` object
     :rtype: requests.Response
 
 post(self, data=None, json=None, **kwargs)
     Sends a POST request.
     
     :param data: (optional) Dictionary, bytes, or file-like object to send in the body of the :class:`Request`.
     :param json: (optional) json data to send in the body of the :class:`Request`.
     :param \*\*kwargs: Optional arguments that ``request`` takes.
     :return: :class:`Response <Response>` object
     :rtype: requests.Response
 
 put(self, data=None, **kwargs)
     Sends a PUT request.
     
     :param data: (optional) Dictionary, bytes, or file-like object to send in the body of the :class:`Request`.
     :param \*\*kwargs: Optional arguments that ``request`` takes.
     :return: :class:`Response <Response>` object
     :rtype: requests.Response
 
 resolve(self)
     Resolve relative path of the path.
 
 with_components(self, *, scheme=<object object at 0x7f96ae1d6a40>, netloc=<object object at 0x7f96ae1d6a40>, username=<object object at 0x7f96ae1d6a40>, password=<object object at 0x7f96ae1d6a40>, hostname=<object object at 0x7f96ae1d6a40>, port=<object object at 0x7f96ae1d6a40>, path=<object object at 0x7f96ae1d6a40>, name=<object object at 0x7f96ae1d6a40>, query=<object object at 0x7f96ae1d6a40>, fragment=<object object at 0x7f96ae1d6a40>)
     Return a new url with components changed.
 
 with_fragment(self, fragment)
     Return a new url with the fragment changed.
 
 with_hostinfo(self, hostname, port=None)
     Return a new url with the hostinfo changed.
 
 with_name(self, name)
     Return a new url with the file name changed.
 
 with_netloc(self, netloc)
     Return a new url with the netloc changed.
 
 with_query(self, query=None, **kwargs)
     Return a new url with the query changed.
 
 with_scheme(self, scheme)
     Return a new url with the scheme changed.
 
 with_suffix(self, suffix)
     Return a new url with the file suffix changed (or added, if none).
 
 with_userinfo(self, username, password)
     Return a new url with the userinfo changed.
 
 ----------------------------------------------------------------------
 Data descriptors inherited from urlpath.URL:
 
 __dict__
     dictionary for instance variables (if defined)
 
 __weakref__
     list of weak references to the object (if defined)
 
 components
     Url components, `(scheme, netloc, path, query, fragment)`.
 
 form
     The query parsed by `urllib.parse.parse_qs` of url.
 
 form_fields
     The query parsed by `urllib.parse.parse_qsl` of url.
 
 fragment
     The fragment of url.
 
 hostinfo
     The hostinfo of url. "hostinfo" is hostname and port.
 
 hostname
     The hostname of url.
 
 jailed
 
 name
     The final path component, if any.
 
 netloc
     The scheme of url.
 
 parts
     An object providing sequence-like access to the
     components in the filesystem path.
 
 password
     The password of url.
 
 path
     The path of url, it's with trailing sep.
 
 query
     The query of url.
 
 scheme
     The scheme of url.
 
 trailing_sep
     The trailing separator of url.
 
 username
     The username of url.
 
 ----------------------------------------------------------------------
 Data descriptors inherited from urllib.parse._NetlocResultMixinBase:
 
 port
 
 ----------------------------------------------------------------------
 Methods inherited from urllib.parse._ResultMixinStr:
 
 encode(self, encoding='ascii', errors='strict')
 
 ----------------------------------------------------------------------
 Methods inherited from pathlib.PurePath:
 
 __eq__(self, other)
     Return self==value.
 
 __fspath__(self)
 
 __ge__(self, other)
     Return self>=value.
 
 __gt__(self, other)
     Return self>value.
 
 __hash__(self)
     Return hash(self).
 
 __le__(self, other)
     Return self<=value.
 
 __lt__(self, other)
     Return self<value.
 
 __reduce__(self)
     Helper for pickle.
 
 __repr__(self)
     Return repr(self).
 
 __rtruediv__(self, key)
 
 __truediv__(self, key)
 
 as_posix(self)
     Return the string representation of the path with forward (/)
 
 is_absolute(self)
     True if the path is absolute (has both a root and, if applicable,
     a drive).
 
 is_reserved(self)
     Return True if the path contains one of the special names reserved
     by the system, if any.
 
 joinpath(self, *args)
     Combine this path with one or several arguments, and return a
     new path representing either a subpath (if all arguments are relative
     paths) or a totally different path (if one of the arguments is
     anchored).
 
 match(self, path_pattern)
     Return True if this path matches the given pattern.
 
 relative_to(self, *other)
     Return the relative path to another path identified by the passed
     arguments.  If the operation is not possible (because this is not
     a subpath of the other path), raise ValueError.
 
 ----------------------------------------------------------------------
 Data descriptors inherited from pathlib.PurePath:
 
 anchor
     The concatenation of the drive and root, or ''.
 
 drive
     The drive prefix (letter or UNC path), if any.
 
 parent
     The logical parent of the path.
 
 parents
     A sequence of this path's logical parents.
 
 root
     The root of the path, if any.
 
 stem
     The final path component, minus its last suffix.
 
 suffix
     The final component's last suffix, if any.
     
     This includes the leading period. For example: '.txt'
 
 suffixes
     A list of the final component's suffixes, if any.
     
     These include the leading periods. For example: ['.tar', '.gz']