Auto import: 1) report for unusual malformed csv and stop execution, report by email 2)....

Auto import: 1) report for unusual malformed csv and stop execution, report by email 2)....

Apr 27, 2017
Alessio wrote
Hello,
I don't know if at the moment there's already an internal check about this but I should test it...anyway I don't think so

1) put the case the supplier had a malfunction, someone made mistakes...rows are not the usual provided etc.

you should analyze the file set for autoimport
eg. columns are not perfectly delimited by the usual delimiter, on a row there are X fields, on other columns Y fields...
obviously this is not good. So, in order to avoid bad imports, you should check this, STOP execution and report this on the email report. Even stop other imports for the same connection until the user checks the files and chooses to unlock this behavior...

2) DO NOT download the same file more than one time
at the moment, it seems to download the same file again and again...check with curl or similar if file is the same (not modified)

3) better parallelize your software. Above all, for the database restore when you get from your database...
I think that could be faster... if you have a 500MB dump (after the unzip...) it's slowing down...
I'm on 8cores and SSD, I don't think that currently is the best you can do...

4) think about a modern layout...for the interface, even if you just change the graphics

thanks
New
1 vote
Vote
Reply
1 Answer
Apr 27, 2017
Alessio wrote
and also, always about parallelism and multicores... I think that's the reason why sometimes the interface is stuck

then, I noticed that you improved the timer for "time spent"...this is good but time spent should be less than before, not only better recognized lol

bye