llmtuner.infogetter.infobase
============================

.. py:module:: llmtuner.infogetter.infobase

.. autoapi-nested-parse::

   Get documents from external sources, perhaps preprocess content

   ..
       !! processed by numpydoc !!


Classes
-------

.. autoapisummary::

   llmtuner.infogetter.infobase.InfoBase


Module Contents
---------------

.. py:class:: InfoBase(interfaceid='', config='', url='')

   Bases: :py:obj:`llmtuner.interface.Interface`


   
   Basic class to retrieve entry information from external source. If you pass a LLMTunerConfig, you can store results in the DB.
   Can be initialized passing a baseurl and an identifier for the source.
















   ..
       !! processed by numpydoc !!

   .. py:attribute:: response
      :value: None



   .. py:method:: get_timestamp(message='')
      :staticmethod:


      
      Get an entry to add to the history of data processing
















      ..
          !! processed by numpydoc !!


   .. py:method:: get_mimetype(filename='')
      :staticmethod:


      
      Get the mimetype from a filename
















      ..
          !! processed by numpydoc !!


   .. py:method:: download(docids)

      
      Download files or webpages to local storage from external URL using its docid
















      ..
          !! processed by numpydoc !!