SAP – How to generate an XSD/IDoc Schema

Below are the steps on how to generate an XSD Schema / IDoc Schema from SAP.

1. In the SAP System in which the IDoc is defined, call transaction WE60 (IDoc Documentation).

2. Enter the name of the IDoc in the Basic type field (or the Enhancement field if your IDoc is an Enhancement of a standard SAP IDoc).

3. Choose menu Documentation, XML Schema.

4. Respond Yes, if prompted to ‘Generate Documentation for Unicode File’.

5. On the resulting screen containing the XSD of the IDoc, choose menu XML, Download.

BizTalk – SFTP Adapter – Missing WinSCPNet Library

Note:

This guide is applicable to BizTalk Server 2016 with Feature Pack 3 Installed.

Version of WinSCPNet library is different once CU5/FP3 have been installed.

Original BizTalk Server 2016 installation requires 5.7.7.

More info here:

BizTalk Server 2016 Prerequisites:

https://docs.microsoft.com/en-us/biztalk/install-and-config-guides/hardware-and-software-requirements-for-biztalk-server-2016

CU5/FP3 Info:

https://support.microsoft.com/en-us/help/4087345/sftp-adapter-improvements-for-biztalk-server

Error:

The Messaging Engine failed to add a receive location “FileSync.SFTP” with URL “sftp://raclede.net/files/*.*” to the adapter “SFTP”. Reason: “System.IO.FileLoadException: Could not load file or assembly ‘WinSCPnet, Version=1.5.4.8265, Culture=neutral, PublicKeyToken=2271ec4a3c56d0bf’ or one of its dependencies. General Exception (Exception from HRESULT: 0x80131500)

How to solve:

  1. Copy the both WinSCP exe and dll (5.13.1) to BizTalk Installation Path (Do not GAC the dll)
    1. WinSCP.exe
    2. WinSCPnet.dll

How to get the exe and dll

  1. Use Nuget in Visual Studio to get the specific version of WinSCP.

Nuget-WinSCP    2.  Once added, just browse to the lib for the dll and tools folder for the .exe

Nuget-WinSCP-InstallPath

 

 

BizTalk – SAP Adapter – RFC_INVALID_HANDLE

All receive location green and by the moment you’ve send your first IDoc this warning occurs:

The adapter “WCF-Custom” raised an error message. Details “Microsoft.Adapters.SAP.RFCException: Details: ErrorCode=RFC_INVALID_HANDLE. AdapterErrorMessage=An exception has occurred on the listener while executing RfcWaitForRequest..”.

To solve this error you’ll need to install the BizTalk SAP Adapter Schema.

Steps:

  1. Create a BizTalk application or use the default BizTalk Application
  2. Right Click Add Resource.
  3. Browse to: \bin\Microsoft.Adapters.SAP.BiztalkPropertySchema.dll
  4. In the options check all.

BizTalkSchemas

BizTalk 2016 – Installing SAP Adapter- Microsoft.Adapters.SAP.SAPGInvoker.dll

After installing BizTalk adapter pack you would expect that enabling the SAP receive location port will just work. Guess what? Error!

Could not load file or assembly ‘Microsoft.Adapters.SAP.SAPGInvoker.dll’ or one of its dependencies. The specified module could not be found.
File name: ‘Microsoft.Adapters.SAP.SAPGInvoker.dll’

To solve this error you’ll need to install the SAP Libraries. Since i have a copy of previous SAP libraries, I’ve just reinstalled those, otherwise you’ll have to download it from SAP Market Place (you’ll need someone from your company that has access to those).

Steps:

  1. Install SAP NCO Libraries:
    1. NCo3018_Net40_x64.msi
    2. NCo3018_Net20_x86.msi
  2. Copy the SAP 3.0 libraries both x86 = 32bit, x64 = 64bit on the correct folder
    1. icudt26l.dll
    2. icuin26.dll
    3. icuuc26.dll
    4. librfc32u.dll
    5. libsapu16vc71.dll
    6. libsapucum.dll

32bit folder = C:\Windows\SysWOW64

64bit folder = C:\Windows\System32

3.  Restart the host instance

 

SSIS 2012 to SSIS 2016 Migration – Arithmetic operation resulted in an overflow

We’ve recently upgraded our SSIS 2012 to SSIS 2016, during the package upgrade it goes without issue. In Visual Studio, there’s no problem running it, however when we use the SSIS runtime we’ve encountered the following error:

“Arithmetic operation resulted in an overflow.”. Possible failure reasons: Problems with the query, “ResultSet” property not set correctly, parameters not set correctly, or connection not established correctly.

The Execute method on the task returned error code 0x80131516 (Arithmetic operation resulted in an overflow.). The Execute method must succeed, and indicate the result using an “out” parameter.

To solve this issue,  I’ve recreated the SQL Task and viola no error!.

 

 

BizTalk – File Adapter – File transport does not have read/write priveleges

Recently, I have to investigate error related to BizTalk – File Adapter. The error that was thrown was:

The Messaging Engine failed to add a receive location “{Receive Location Name}” with URL “{RecieveLocationPath}\*.*” to the adapter “FILE”. Reason: “File transport does not have read/write privileges for receive location “{RecieveLocationPath}\”. “.

I think the error was obvious, and these are the steps I’ve performed to investigate:

  1. Checked if path exists (duh!).
  2.  Checked the Share Folder settings and Security Settings (looks ok!)
  3.  Double checked which account is accessing the folder in BizTalk Admin Console. Since there was no account specified in the Authentication Tab, then I assume that this is using the default account of the Host Instance.

BTS-FileAdapter-AuthenticationTab

4. Re(enable) the Receive Location (immediately shutdown)

5. Restart the Host Instance, repeat step 4. (doesn’t work)

6. Re-enter the password of Host Instance Service (using services.msc) and restart and repeat 4 (no use!).

6. Reboot the Server (still doesn’t work).

Hmm, weird huh?

What i did next is to go to Shared Folder Management (in the server where the folder is hosted). This can be access using Run-> mmc -> Add/Remove Snap-In -> Share Folder.

From the menu, go to Open Files.

MMC-ShareFolder

What i see is: there are 2 different service accounts of BizTalk connecting to the same share folder server. Hmm.. So i changed one of the Receive Location to use the same account as the other one and repeat Step 4.  After this step, the receive locations are now green again!.

It looks like BizTalk can’t connect to a Share Folder Server using 2 different accounts.

 

 

 

 

 

Cryptocurrency and BitCoin (BTC) Mining

It was roughly 2 years ago when I’ve started looking at BitCoin(BTC) and first impression was: WhoTH will be willing to spend 400 EUR for 1 BitCoin? Fast forward 2018, last check it was 13,500 EUR.  2 years ago, for every 1 EUR investment it will be x33.75 (3375%).

At this moment, I’ve invested into different virtual cryptocurrencies such as LTC, ADA and ETN. Hopefully this will go to the MOON.

Also invested a coin called PACCoin for 30 EUR a week ago and it’s now worth 300 EUR. That’s 1100% increase in just a week. Should’ve invested more.

Lastly, I’m looking into different options of BitCoin mining, from buying your own AntMiner S9, this roughly costs (3,000 EUR) or assembling your own 6GPU rig, will roughly cost 2,500 EUR both options requires a lots of upfront investment. Another option is to do cloud mining. I’ve looked at numerous cloud mining provider, Genesis doesn’t have a contract for SHA256 mining therefore next possible option is to go Hashflare, this company according to google search is also producing bitcoin mining machine. I’ll provide an update in the future based on my experience using this cloud mining provider.

If you want to try cloud mining click BitCoin Cloud Mining – Hashflare.io. The link contains my referral code 🙂

Now some terms in cryptoworld:

FOMO = Fear of Missing out

MOON = This refers to a crypto’s upward momentum as it keeps climbing in price, as in, The price of this coin will one day go to the moon!”

 

SSIS – Use timestamp to detect changes

Last week, a solution was required to build an SSIS solution that can pick up all changes from a database using the versionnumber (timestamp) and sync the changes to an external database using the timestamp of the database. This is quite easy to achieve by:

  1. Step 1, retrieve the previous timestamp. Storing the previous timestamp can be easily achieved by create a file using File.WriteAllText method.

 Dim runLastFile = System.IO.Path.Combine(currentPath, “LastRun.txt”)     System.IO.File.WriteAllText(runLastFile, lastRun)

I know it’s in VB but you can easily convert it to C#, since it’s an old solution i didn’t bother to convert it. If the file is not yet existing, you need to create a logic to get the lowest timestamp. Like this:

declare
@minVersion binary(10),
@minVersion_str nvarchar(42)

SET @minVersion = ( SELECT MIN(timestamp fieldName) FROM {sourceTable} )
SET @minVersion_str = upper(sys.fn_varbintohexstr(@minVersion))
SELECT @minVersion_str as LowestVersion

2. Step 2, get the current timestamp from the source database by using MIN_ACTIVE_ROWVERSION, the challenge however is to store this as string/text so the SQL query can be constructed later on in the process. After 15 minutes of googling I was able to find it somewhere (credit to the forum, lost the link).

declare
@currVersion binary(10),
@currVersion_str nvarchar(42)

SET @currVersion = MIN_ACTIVE_ROWVERSION()
SET @currVersion_str = upper(sys.fn_varbintohexstr(@currVersion))
SELECT @currVersion_str as CurrentVersion

3. Step 3, construct the SQL query using the previous timestamp. Something like this:

                         SELECT {sourceColumn}
FROM {sourceTable}
WHERE {timestamp fieldName}  >= @previousTimeStamp

4. Step 4, if the process is successful don’t forget to store the current timestamp.

5. Step 5, start from step 1.

Key to successful BizTalk Deployment

It’s been a while since my last blog, but IMHO this is one of important blog that I’ve ever made that can be useful pointer for every aspiring developer / companies that will/already implement application integration. Below contains a list / summary of best practices I’ve acquired in my more or less 10 years working with BizTalk, SSIS, WCF and .NET integration (primarily Microsoft Technology) but this I think is also applicable to other technology.

Design / Development:

#1 K.I.S.S (Keep it simple, stupid). Don’t get me wrong, I like complex integrations, from  multiple message correlations, requirement to aggregate certain line items with certain condition in the message to having a direct binding with delivery notification set to transmitted (multiple send port / send port groups, this will result in some zombie messages). Anyone from novice to expert BizTalk Developer can build a solution, it can be done & solve  in different ways but the key here is providing a simple and effective solution for a complex integration.

#2 Isolation with maximum reusability. Isolation for me means that the application can be deployed without having too much dependency to other BizTalk artifacts that are shared. During the design it’s always a struggle on how to seperate artifacts, on which solution / projects it should be placed to avoid having to uninstall all referencing application before you can refresh the dependency (GACing works in some situation but not for all, you’ll be surprise how BizTalk deal with it internally). Once identifying or isolating a certain integration flow whenever there’s a new requirement, we should always check whether it fits on already existing solution (extend) without having to build a new integration flow. in this case we will have maximum reuse of existing solution.

#3 Consistency. Every solution that will be build should look as if it’s build / coded by one person. Having a naming convention on BizTalk artifacts, namespace, solutions, project & folder structures should be define first. Based on experience providing a proper name is always a challenge since if you define it incorrectly, it will be a costly mistake later on.

#4 Nuget – all common code / components / libraries should be referenced from a local nuget repository. Having a local nuget repository not only simplifies referencing dependencies but also saves you once you start using TFS Build Server (autorestore feature). This means that shared assemblies is not required to be checked in together with the source codes.

#5 .NET over XLST in complex mapping  Whenever there’s a requirement for complex mapping like grouping, aggregation, unique numbering over a certain combinations or constructing a hashtable in the mapping, I always go for .NET component. Using .NET component it is easier to maintain and tested (unit testing + mockup) vs doing everything in XSLT (in which can be forgotten in time).

Build / Deployment:

#1 Release Pipeline – It’s important to define strategy on how you would build and be able to do ‘Build once’ and deploy to multiple environments. In our case, part of our strategy is to minimize branching and always build from MAIN. In TFS, we’ve implemented a Gated Checkin which means that every checkin will trigger a release pipeline. During the checkin, a build will be triggerred (MSBuild and BTDF build), it will run all unit test and copies the binaries to drop folder (only If all steps are successful). From this point the Ops can just enter a command (application name) and environment and then the automated deployment will take over..

#2 Configuration over Code for normal deployment most of time you would see that a solution contains a powershell script for deployment, this is not bad but i think this only works on sample codes or tutorials but a problem in Enterprise deployment. For this you need a good deployment framework (compilation of scripts) to do the deployment and during the deployment preparation just replace the values in the templates like servernames, shared folders and etc.  I’ve posted some blogs earlier how to do this:

Centralize Powershell Repository:

https://randypaulo.wordpress.com/2012/02/27/powershell-centralize-powershell-script-modules-repository/

Visual Studio 2012 – Install Shield LE Basic Tutorial

Starting Visual Studio 2012, the Setup project for Visual Studio is not supported anymore and the only free alternative is to use the Install Shield Limited Edition (LE).

In this post, I will describes the steps to successfully:

A. Deploy a project to GAC (Global Assembly Cache)

B  Generate/pacakage Single MSI for installation

 

Steps:

1. Start by downloading the Install Shield here. And installing it.

2. Next is to create a new install shield limited edition project.

3. Follow the wizard, and in the step 2, right click the destination computer and select [GlobalAssemblyCache]

4. Drag and drop the Primary output from the Solution (top pane).

5. Go to every item in the [GlobalAssemblyCache], right click Dependencies from scan at build. Be sure to unselect other reference assemblies so it will not be registered in GAC.

Single MSI Generation:

6. Now to create a single MSI setup, Go to step 6 (Releases), Select the Single Image. In the build tab, Set compression = compressed, and on the Setup.exe tab set the Setup Launcher to No.

7. Right click the Solution, then select Configuration Manager, in the InstallShield project set the configuration to Single Image. Then Right click Install Shield project – > Build.

8. Go to Install Shield Project – > Express – > Single Image – > DiskImages -> Disk1. Viola. the MSI can be found there.