Thursday 16 July 2020

Working with JSON and MS SQL server


    JSON (JavaScript Object Notation) is an open standard file format, it is a text based data exchange medium, now a days most of the API gateway's returns JSON output, and other applications consume it,  if we talk about SQL and Database area many places we get requirement to read JSON data, load that in a database to process further for OLAP application's.
     Microsoft SQL server 2016 and above version's support JSON transactions with multiple helpful functions, which helps us to make our life easy. in this article i am trying to simplify how to work with JSON and MS SQL server.

   Before that just look into this diagram to understand importance and uses of JSON.

lets begin with Sample JSON Code, you can find more details of JSON from https://www.json.org 
Here is a sample JSON information data, with multiple attributes. 

[
  {"name":"anup",
  "company":"CYZ",
  "city":"pune",
  "skills":["SQL","MSBI","C#","power BI"]
  },
  {"name":"vishal",
  "company":"IXZ",
  "city":"hyderabad",
  "skills":["SQL","MSBI","reports"]
  },
  {"name":"moin",
  "company":"DXZ",
  "city":"hyderabad",
  "skills":["SQL","MSBI","C#","informatica"]
  }
]

now the first question comes in our mind is how to read this in SQL server and display it in tabular format? for this we have a Function called OPENJSON() similar like OPENROWSET, OPENXML in SQL server, lets see a simple way to read it.

1) OPENJSON() 

declare @json nvarchar(1000);
-- Sample JSON string
set @json = '
[
  {"name":"anup",
  "company":"CYZ",
  "city":"pune",
  "skills":["SQL","MSBI","C#","power BI"]
  },
  {"name":"vishal",
  "company":"IXZ",
  "city":"hyderabad",
  "skills":["SQL","MSBI","reports"]
  },
  {"name":"moin",
  "company":"DXZ",
  "city":"hyderabad",
  "skills":["SQL","MSBI","C#","informatica"]
  }
]'
-- selecting JSON Text in tabular format
select * from OPENJSON(@json)

but output of above query is not exact tabular format so we need to read this column wise.(Column names mentioned in with clause should match with json attributes)

select * from OPENJSON(@json)
with(
name varchar(10),
company varchar(20),
city varchar(10),
skills varchar(10) '$.skills[1]',
skills varchar(10) '$.skills[2]',
skills varchar(10) '$.skills[3]',
skills varchar(10) '$.skills[4]'
)

-----------------------------------------------------------------------------------------------------------------------

2) Read JSON_VALUE consider we have inserted JSON as a text in column and we have to extract values out this then we have JSON_QUERY, JSON_VALUE Functions in MS SQL server.

--Inserting JSON Text as column value in table
Create table #jsontbl (id int identity(1,1), jsoncol varchar(3000))
insert into #jsontbl(jsoncol) Values (@json)
select * from #jsontbl

see the query to read values out of json text and to validate json text

-- Example 1
select
JSON_VALUE(jsoncol,'$[1].name') as 'Name',
JSON_VALUE(jsoncol,'$[1].city') as 'City',
JSON_QUERY(jsoncol,'$[1].skills') as Skills
from #jsontbl
where ISJSON(jsoncol)>0

Updating JSON

-- Updating JSON Value
DECLARE @json1 NVARCHAR(MAX);
SET @json1 =
'{
  "info":
        {   "address": [
                        {"town": "Pune"},
                        {"town": "Hyderabad"},
                        {"town": "Delhi"}
                       ]
        }
 }';
SET @json1 = JSON_MODIFY(@json1, '$.info.address[2].town', 'Mumbai');
Print @json1;

-----------------------------------------------------------------------------------------------------------------------
3) Exporting SQL Data in JSON Format. this is again a very important if we want to give data feed to other application in a json format. use - JSON PATH 

select [id], [event],[operator],[executionid]
from MyDB..sysssislog
for JSON PATH

-----------------------------------------------------------------------------------------------------------------------
4) Open JSON text file from physical location to SQL server and process further. we can write query something like this.

-- Read JSON Data from txt file
Select * from OPENROWSET(BULK 'C:\Data\jsondata.txt',SINGLE_BLOB) tbl

-----------------------------------------------------------------------------------------------------------------------

i tried to bring here all JSON topics and functions from SQL server 2016 and later versions with more simpler manner so one can expand this code further as per your requirement, Happy coding !!!

Thank you !






Tuesday 19 May 2020

Call REST API through SSIS package

    Consuming REST API in SSIS is not as easy as Consuming Web Service, we have Web service task in SSIS, but there is no specific task for Calling API, hence we will use c# code to consume API inside Script task.
  below are the step by step methods for calling API

1) Use Script task for API calling, declare a variable to hold API URL.

use script task for API call


2) Go to script task, there is default c# code. that does not have API references, hence we need to add reference to System.Net.Http.

Add reference to system.net.http


3)Write a c# code to consume Web API, we can write success or failure logic based on API result.

c# code for API calling


add below code inside your main() block. use namespace which we added in reference.

using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
       
public void Main()
{       
            try
            {
                string serviceUrl = Dts.Variables["User::API"].Value.ToString();
                string GUID = Dts.Variables["System::ExecutionInstanceGUID"].Value.ToString();
                
                HttpClient client = new HttpClient( new HttpClientHandler { UseDefaultCredentials = true }) ;
                client.BaseAddress = new Uri(serviceUrl);

                // Add an Accept header for JSON format.    
                client.DefaultRequestHeaders.Accept.Add(
                    new MediaTypeWithQualityHeaderValue("application/json"));
               
                string APIUrl = string.Format(serviceUrl , GUID);
                var response = client.GetAsync(APIUrl).Result;
                if (response.IsSuccessStatusCode)
                {
                    var result = response.Content.ReadAsStringAsync().Result;                    
        
                    Dts.TaskResult = (int)ScriptResults.Success;
                }
                //Dts.TaskResult = (int)ScriptResults.Success;
            }
            catch (Exception ex)
            {
                Dts.TaskResult = (int)ScriptResults.Failure;
            }
}

4) we can also use API calling c# code in Script component as Source or Transformation or Destination. also.
 
Thanks !!

Monday 10 February 2020

Understand the Private Equity Domain


What is Inside Private Equity Business?
    Private equity business is wonderful business, or one can say its brain game, or I can even say, let your excess money work for other and you also!, I have been developing applications for PE since 
5-6 years. as a Software developer, I have developed different types of business intelligent reports, dashboards, SQL & ETL development for PE Client. as well as I have interest in Stocks and Equity investment, 
   software engineers or professionals who are working in PE domain they must understand this domain, to give more productive output and grow faster in this industry. I tried my best to put Private Equity domain more simple way to understand, even for beginners. We will start from the basics first.

I have divided above article in 5 Topics for better understanding.
1)     Equity Basics
2)     Inside Private Equity business
3)     Types of Private Equity
4)     Fund Operation’s and Security Master
5)     Well known examples, Deals of PE business.

I haven’t written more elaborated descriptions, just to avoid confusion or bouncer balls! my theme is Small and easy to understand.

------------------------------------------------------------------------------------------------------------------------
Lets Start point to point. 
------------------------------------------------------------------------------------------------------------------------

1)     Equity Basics 

 - What is an Equity? First question, this is the first step we should know. We can say an Equity is the wealth of someone!
   Let’s discuss some Basic points of  Equity.
  • Equity
  • Legal Entity
  • Share
  • Share Capital
  • Security
  • Investor & Funds
  • Bonds & Debentures
 1)   Equity = Asset Value – Liabilities
Example-  
Asset (Flat 40L + Cash 5L + Car 5L) – Liability (Home Loan 30L + Car Loan 8) = +12Lakh is Equity of Person A.

Asset (Flat 40L + Cash 2L + Car 5L) – Liability (Home Loan 35L + Personal Loan 5 +Car Loan 8) = -1Lakh is Equity of Person B. 

Asset (800cr) – Liability (300cr) = 500Cr is an Equity of K Company.

2) Now what is Legal Entity? – It’s a company or a legal body.

3) Share Capital – Share is a divided equity in a Legal Entity, and Share capital is the total amount raised by the company in sale of shares.

Now we understood Share Capital, but to raise that capital from investors we should give back something trusted contract or government paper or something Security, right?

4) What is Security - A security, in a financial context, its a certificate or other financial instrument that has a monetary value, and can be traded. Securities are generally classified as equity securities, such as stocks. and debt securities, such as bonds and debentures.

5) And who buys all this? an Investor -The investors are those who gives money, in return of some security, like shares, bonds, Funds. (Funds could be an amount collected for special investment purpose)

6) Bonds – bonds are fixed payable contract between investor and Legal Entity. It’s a Security, investors pays money against bonds security.

7) Debentures – are a long term security with fixed rate of interest earning issued by a company.

------------------------------------------------------------------------------------------------------------------------
Now will go further inside of Private Equity business... 
------------------------------------------------------------------------------------------------------------------------

2) What is Inside Private Equity Business?

- a Term private equity is very broadly classified and customized, years on year however,
 “Generally Private Equity is, any equity investment in a company which is not quoted on a stock exchange.” it’s a basic definition.
But we can define it some more ways. like,
Something publicly listed company is taken privately and unlisted,
Or buying big shares in Listed Company as part of portfolio,
Or Investment in Real estate project (IT parks, Housing Project, Road Bridge building Projects, SEZ),
Or the acquisition of other company,
Or the investment in startups (like Snap Deal, OYO, OLA, Swinggy, Byju’s),
Or investment in good but loss making company and convert it into profit making (like Flipkart, Paytm).

Till here we understood that PE business collect money from investors and Invest in different businesses, Shares, Buy a Company, gives loan. now we will see different ways of investments PE company does.
basically Investments are classified in 2 ways.

•  Fund Investing – is something collecting money from multiple investors.

•  Direct Investing - are those who manage this funds are also called GP who does Direct 
       Investing in multiple company buys shares etc., GP have full rights of investment and investment
       strategies

•  LP, GP, OP – are the partners, will understand the partners of company, 
     • LP (Limited Partner) are the investors who invest money in Fund.
     • GP (General Partner) are the Fund managers, PE company.
     • OP (Operational Partner) are those who operate funds of other big fund houses, are generally called as Hedge Funds.

------------------------------------------------------------------------------------------------------------------------
So far we got a definition of Private Equity and GP, LP, OP, but here is the question, what does PE company does with collected money? and how do they make money? are there any types? Lets see that...
------------------------------------------------------------------------------------------------------------------------

3) Types of Private Equity Business

Private equity funds generally fall into two categories. 
Venture Capital and Buyout or Leveraged Buyout.

Venture Capital  
“Venture Capital is nothing but providing capital to Startups new Ventures with unique business ideas”
Typically invest in small early stage and emerging businesses, that are expected to have high growth potential or have innovative Ideas, although venture capital funds carry risks from investing in unconfirmed emerging businesses, they can generate extraordinary returns or losses.

Buyout or a Leveraged Buyout 
“Buyout is something majority share of the stock of the company is acquired”
Buyout - happens in large companies not in small startups which venture capital does,
Leveraged Buyout - A leveraged buyout is the acquisition of another company using a significant amount of borrowed money (bonds or loans) to meet the cost of acquisition

Above two are types of Private Equity business, but I would like to add another area, kind of master type called as Hedge Fund.

Hedge Funds – are the fund collected from big players, or fund houses, or Institutional investors and follows route of either venture or Buyout investments.

Here are some basic differences of Buyout and Leveraged Buyout.

Venture Capital Buyout or Leveraged Buyout
Small enterprise, Young companies, even start-upLarge or very large enterprise, not startups
Bank debt almost never usedBank debt almost always used
Converts startups to bigger profitable company Increases the profit of company by reducing debts 
Exit by selling company Exit by selling Equity



------------------------------------------------------------------------------------------------------------------------
As of now we have learned many things in PE business, from basics to the types of PE business and partners, now we will go some more deep in Operations, how does the operations happen in PE.
     This is more interesting subject for Software Engineers who program PE needs, develop software for PE, designs Database, ETL and reports....
------------------------------------------------------------------------------------------------------------------------


4) Fund Operations and Security Master.

The operational flow is as shown below.
1) Investors Invest money into Fund, (Vehicle) Then
    --> Fund (Legal Entity)  Legal Entity invest that money in  --> Interesting Deals
    --> Deals can have multiple Positions
    --> Now Fund house pays money to Issuer (Seller)
    --> for that Issuer gives back Security like Bonds, Property Papers, Shares.
    --> all above business Transaction will captured for Audit.

I have a detailed diagram to illustrate the entire process below, link above flow with diagram to understand in details.


Looking at above diagram you can understand detail picture of PE business flow. We as a software developer develop application around this, business flow. And database designs are also similar across all PE business. There are multiple products used to drive the PE business
     SunGard Investran is one of the popular software, now its FIS Investran.
Alternative popular application for Investran are
1) Juniper Square, 2) IMS (Investor Management Services), 3) Dynamo,  4) VTS, 5) Stessa, 6) CoStar Investment Analysis, 7) MRI Investment Management, 8) Lease Eagle

#Investran #JuniperSquare #IMS #Dynamo #VTS #Stessa #CoStarInvestmentAnalysis #MRI
 #LeaseEagle

------------------------------------------------------------------------------------------------------------------------
This is what the Private Equity business, i have tried to cover maximum topics in this article with minimum words. however, without knowing best examples one cannot understand PE very well,
lets see some of the popular PE deals....
------------------------------------------------------------------------------------------------------------------------

5) Well known Deals of PE business

Most of us knows these deals but unaware that it’s a PE deal.

1) Venture Capital Deals Example - Paytm, Future Retail, Flipkart




    Soft bank, worlds one of the biggest Private Equity Firm, India Deals.



Blackstone’s Leveraged Buyout / Buyout Deals of taking over of bad Loans 




Few of biggest Buyout deals like Mphasis, IBS, SH Kelkar.


Most of the things i have covered here in this article, If any thing is missing in this article, please add your knowledge share from comments, suggestion, ideas 

Thank you !