=============
Data Import
=============

With *jmb.core* is possible to import data in your project database by processing a xls file or *csv* line by line
and than create or edit your models.


Create do method
----------------
First step is declare a class that extends :ref:`XlsFile <XlsFile_class>` or :ref:`CsvFile <CsvFile_class>` (which inherit
from :ref:`Sheet <Sheet_class>`) in which you must overwrite **do** method where you can process the file line by line.
*do* method use follow parameter:


        :row: xls row object. Row element can be taken with row.<column_name> where <column_name> is the unicode
                    and case-sensitive head column (at first row).
        :kw:  dictionary of mapping column->model.field (if fields_map and relative model is set kw is
                    automatically fill).
        :j:   row number.


Process cell by cell
********************
Follow code is an example of how to override the *do* method for import an xls file:
::

    from jmb.core.admin.import_data import ImportData

    class MyImportData(XlsFile):
        def do(self, row, kw, j):
            customer = Customer.objects.create(company_name=row.Company_name, vat = row.Vat)


Process with a mapping
**********************

If you are importing an xls file where the column coincide with all or with parts model field, you can map column
head with model field like this:

::


    from jmb.core.utils.data_import import XlsFile

    class MyImportData(XlsFile):
        fields_map = {
            'code' : 'ancode',
            'description' : 'andescr',
            'foo' : 'anfoo',
          }
        def do(self, row, kw, j):
            Contact.objects.get_or_create(code=row.ancode, defaults=kw)

    x = MyImportData('anagrafiche.xls', auto=True)
    x.read()

In *fields_map* dictionary the keys are model field name and the values are column head (the space are replaced with _).

Define a custom form
--------------------

*jmb.core* provide a default form that already contains a simple form with upload widget. This step is only necessary if
you want customize your import form and use cleaned_data in *do* method.
Custom form is simply a form that extend :ref:`ImportDataForm <ImportDataForm_class>`:

::

    from jmb.core.admin.forms import ImportDataForm

    class MyImportForm(ImportDataForm):
        """
        This class is not required, if you not declare a custom form you will use default form ImportDataForm.
        """
        import_option = forms.CharField(
            max_length=50,
            required=False,
        )

Form values will be available in your XlsFile extended class as opt instance attribute

::

    from jmb.core.utils.data_import import XlsFile

    class MyImportData(XlsFile):
        def do(self, row, kw, j):
            import_option = self.opts['import_option']
            ....

    x = MyImportData('import.xls', auto=True)
    x.read()




Define a view
-------------

The view must be extend :ref:`ImportDataView <ImportDataView_class>`:

::

    class MyImportView(ImportDataView):
        """
        Custom view for import file, for a proper configuration you must set:

            import_class = MyImportData

        where MyImportData is a class that define a do() function.
        If you can set (but not is required) a custom form to use for import:

            form_class = MyImportForm
        """
        import_class = MyImportData  # Mandatory
        form_class = MyImportForm    # Not Mandatory



Configure url
-------------

Last step is define a step that call your *MyImportView*:

::

    urlpatterns = patterns(
        ...
        url(r'import_data/$', views.MyImportView.as_view(), name='import_data'),
    )



Use a command
-------------

An alternative to *form + view* solution is a command solution. Below is a simple example of how to create a command to import a customer::

    class ImportCustomer(XlsFile):
        def do(self, row, kw, j):
            customer = Customer()
            customer.company_name = row.Company_name
            customer.vat = row.Vat
            customer.save()

    class Command(BaseCommand):
        option_list = BaseCommand.option_list + (
            make_option('-f', '--file', dest='filename', help='Read file in argument', default=None),
        )

        output_transaction = True

        def handle(self, *args, **options):
            filename = options.get('filename')
            if filename is not None:
                plants_to_import = File(open(filename, 'r'))
                import_data = ImportPlant(
                    file_contents=plants_to_import.read(), auto=True)
                import_data.read()
            else:
                print "please indicate file to import with -f <file_name>"