Showing posts with label Nightly ETL process. Show all posts
Showing posts with label Nightly ETL process. Show all posts

Monday, 15 January 2018

Reducing long query times through data compression and sql reduction

Nightly process completion at Afro Incorporation
Irfan, Production Manager, met Sohail(IT Head) in the meeting room.
Irfan: what happened champ? Look worried?
Sohail: It is the nightly report process that begins at 12 am. It is not completing by 9 am. Suneet(IT Manager) is asking for a new server costing Rs. 25 lakhs. Currently my SLA(service level agreement) is not been met.
Irfan: what will happen with the new server?
Sohail: will speed it up. The management is saying that there is no budget left for this year for IT.
Irfan: Now what?
Sohail: am stuck. How is it going for you?
Irfan: very nice. Since the new Japanese(Takashida) came in. He is removing the constraints in production, reducing inventory and increasing throughput. Why do you start the daily process at 12 am. Why not before?
Sohail: because complete data comes by midnight.
Irfan: say a transaction has happened at 11 am in the morning, then  as per the production team, it is in stock for 13 hours since the other data has not come. Am I correct?
Sohail: correct.
Irfan: Takashida would pre process the data immediately and remove the constraint. He would then store the same in a Work in progress area so that the final fitment is quick.
Sohail: basically work for 24 hours instead of 9 hours.
Irfan: correct. Remove the time constraint. But how would you store the work in progress output?
Sohail: I had this consulting team from M&N BI come up to me that it could store the Work in progress output in a separate database by pre processing the business logic
He gave me this link for data consolidation, datatransformation, data cleansing and even Business Intelligence from his website. The company was saying that even advanced analytics and resolving complex inventory issues should be possible by using its products.
What do you suggest?
Irfan: Let us see what Wasim(CEO) says.

An example of what his team has done for another firm is attached herewith:
http://mndatasolutionsindia.blogspot.in/2018/01/query-reductionimplementation-example.html


If you have the same problem as Sohail, contact the following
By:
Name: Apoorv Chaturvedi
Website: www.mnnbi.com
Phone: +91-8802466356


Initial and Incremental data Load Template by M&N Business Intelligence-SAP Data Services

  INCREMENTAL LOAD/CREATION OF DIMENSION TABLE LOGIC AT SAP DATA SERVICES END CLIENT In this particular document we shall be looking at the ...