data import

Importare dati

Questo modulo fornisce alcune classi per facilitare l’importazione di dati da file excel e csv Si suppone che ogni file sia composto da fogli Sheet

Un esempio di importazione sarà:

from jmb.organization.models import Contact
from jmb.core.utils.data_import import XlsFile

class Anagrafiche(XlsFile):
    fields_map = {
        'code' : 'ancodice',
        'description' : 'andescri',
        'foo' : 'antipcon',
      }
    def do(self, row, kw):
         # modi alternativi di referenziare i dati
         Contact.objects.get_or_create(code=row.ancodice, defaults=kw)
         #Contact.objects.get_or_create(code=kw['code'], defaults=kw)

x = Anagrafiche('anagrafiche.xls', auto=True)
x.read()

field_list

un metodo semplice di dare il nome alle colonne è di impostarlo con l’attributo Sheet.field_list, le colonne possono poi essere mappate in field names tramite Sheet.fields_map

class jmb.core.utils.data_import.Sheet[source]

Abstract sheet

do(row, kw, j)[source]

implement here your import

Parameters:
  • row – a namedtuple. Names are the names in the original columns
  • kw

    a dict suitable to feed defaults keyword attribute of get_or_create if fields_map is provided:

    def do(self, row, kw):
        Contact.objects.get_or_create(cod=row.cod, defaults=kw)
    
  • j – integer 1-based counting read rows
field_list = None

a list of field names. Optional. When present it will be used to name each column instead of looking at the column header. Field name here is the row attribute (eg: row.first_name)

fields_map = None

a dict mapping field names to column headers, as an example, if field_name has {‘first_name’ : ‘Nome’} it means that read() will set:

kw['first_name'] = row.Nome
handle_exception(e, row, kw, j)[source]

Handle exception of the execution of do()

read(limit=0, start=0)[source]

Loop on all rows of a sheet

Parameters:
  • start – start reading at line nr. start
  • limit – limit number of rows to read to limit

xls

class jmb.core.utils.data_import.XlsFile(filename=None, file_contents=None, auto=False, sheet_index=0)[source]
get_cell_value(cell)[source]

Return the best guess for the correct value

get_rows(sheet_index=0, sheet=None)[source]
Parameters:
  • sheet_index – the index of the sheet to read
  • sheet – a possible sheet object as returned by sheet_by_index
get_sheet_as_list(sheet_name=None, all_sheets=True)[source]

get named sheet or all sheets

get_tuples_from_row(Row, row)[source]
Parameters:
  • Row – typename
  • row – iterable
xldate2date(cell)[source]

Returna un datetime.date dal valore della cella

xls2csv()[source]

Really poor version of csv file

csv

class jmb.core.utils.data_import.CsvFile(filename, auto=False, delimiter=';')[source]
get_tuples_from_row(Row, row)[source]
Parameters:
  • Row – typename
  • row – iterable
class jmb.core.utils.data_import.Sheet[source]

Abstract sheet

clean_field_name(value)[source]

Clean each column heading to make it a suitable field_name. By default strip empty spaces, non ascii chars, parenthesis :param value: the column header

do(row, kw, j)[source]

implement here your import

Parameters:
  • row – a namedtuple. Names are the names in the original columns
  • kw

    a dict suitable to feed defaults keyword attribute of get_or_create if fields_map is provided:

    def do(self, row, kw):
        Contact.objects.get_or_create(cod=row.cod, defaults=kw)
    
  • j – integer 1-based counting read rows
field_list = None

a list of field names. Optional. When present it will be used to name each column instead of looking at the column header. Field name here is the row attribute (eg: row.first_name)

fields_map = None

a dict mapping field names to column headers, as an example, if field_name has {‘first_name’ : ‘Nome’} it means that read() will set:

kw['first_name'] = row.Nome
get_fk_as_dict(model, value='id', key='name')[source]

Return a dict of all values of a (possible) fk as dict

Parameters:
  • model – the django db model
  • value – the field to use as value of the dict (default id) Note: will be lowered
  • key – the field to use as key of the dict (defaykt name)
handle_exception(e, row, kw, j)[source]

Handle exception of the execution of do()

nullable_field_list = None

Fields listed in this list will be set None if empty specifically needed to prevent unique problems

read(limit=0, start=0)[source]

Loop on all rows of a sheet

Parameters:
  • start – start reading at line nr. start
  • limit – limit number of rows to read to limit
setup()[source]

Setup env possibly used in self.do

Previous topic

Dashboard

Next topic

Postgresql

This Page