Wednesday, 11 April 2018

HTTP 400 bad request response

Problem:  I have an old SharePoint 2013 application that is partially loading.  This is only happening for 1 user out of thousands and occurs on Chrome and IE.  I can see some in the IE developer tool bar that some requests are showing 200 responses and some are showing 400 responses from the web servers.  I am load balanced and all WFE's are showing the 400.

Initial Hypothesis:  Only 1 user has the issue.  Some URL requests work and other are malformed (return 400 errors) on the same WFE.   User on different machine still fails.  Using different browser user still fails.  The user is forming a malformed request.  It appears to be a problem with the specific user to a specific site collection and is likely to be the HTTP Header request.
Using the browser settings/Fiddler or Dev toolbars get the error details e.g. HTTP 400 – Bad Request - The size of the request headers is too long.

Possible Resolution: Look at the request header, it may be too long for the WFE to handle.  As making the header smaller is generally not an option, look to increase the the size of the requests IIS allows for HTTP requests (HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\HTTP\Parameters).  As this is production issue and I can't replicate to a lower environment, I need to use a host entry to get my offending user to only be accessing a single WFE where the fix is applied.  By using the NLB and updating IIS, I can ensure the fix works without disrupting my user base.
See https://www.grouppolicy.biz/2013/06/how-to-configure-iis-to-support-large-ad-token-with-group-policy/



Wednesday, 28 March 2018

TFS Scrum for SharePoint Projects

Great information on TFS for Scrum, Agile & CMMI from Microsoft.  My preference is to use Scrum with a couple of twists from Agile and external tooling.
https://docs.microsoft.com/en-us/vsts/work/work-items/guidance/choose-process

Tip:  I use User Stories extensively in scrum and with TFS all the testing and automation fits in brilliantly.  CI/CD is a chouch between TFS build and TeamCity.  Also my acceptance criteria is always written using gherkin language to ensure consistency.

Below are a few posts that are a couple of years old outlining Agile and Scrum for SharePoint projects:
Agile for SharePoint
Scrum for SharePoint - Part 1
Scrum for SharePoint - Part 2

Friday, 23 March 2018

An approach to building transactional systems in SharePoint

Overview:  It is common with modern SharePoint development to store transaction high volume data inside a SQL database and expose the application data using WebAPI or a WCF.  The application, e.g. SPA's, Angular or SharePoint pages itself merely calls the "web service" and viola you have an application that is fast and complex with the SharePoint world.


Problem:  When the WCF/WebAPI goes to the database we use a single account (single account principal).  This is an age-old problem in BI, and web applications.  The solution options are to have the security in the database, or each user needs to have a SQL login. 
Initial Hypothesis:  Generally in the last 20 years the majority of application go for the single principal data access approach.  This means there is no logging in SQL natively and you need to pass in the user's context (usually a username or email address).
My Solution:  I use the single access account principal, so I connect using the same account (either encrypt or use something like Azure Vault, in the old days this was the web.config entry with a username and password.  Each request needs to be unique so I pass in the username with the request, and my queries have users and roles and using these relationships I can validate that my user has rights to perform CRUD operations.  I am a huge fan of SQL 2016, as its performance is miles ahead of SQL 2014 and it supports "TemporalTables".  Now with other older SQL instances, you could build your own database logging (tomb tables is what I use to refer to it as).  Worth noting is that Entity Framework does not support Temporal Tables yet, but surely this will come. 
This solution provides a flexible, fast HA (assuming AOAG) transaction secured system with non-repudiation and full logging.  Overall I find this a great approach to building out complex solutions for my clients. 
This approach also provides a easy re-usable API that can be used to allow other applications and business partners to integrate with the solution.  It also allows for a mobile application UI to be easily added as the API are already in place.

Saturday, 10 March 2018

SharePoint Tooling 2018

Work in Progress...
On a development machine I have a ton of tools depending on the development approach and technologies used.  This post lists tool I commonly use as of March 2018:
SharePoint Tools:
  1. SharePoint Inspect
  2. SharePoint Designer
  3. Visual Studio
  4. SharePoint Search Query Tool (CodePlex now PnP)
Design Tools:
  1. Balsamiq (My favourite) - Screens and interaction flow
  2. Microsoft Blend - Screens and interaction flow
  3. Visio - Architecture
  4. Access for ERD design
Other Tools:
  1. SnagIT - Basic video recording with audio and annotate screen shots.
  2. Office & OneNote
Networking:
  1. Wireshark
  2. Fiddler
  3. DeveloperTool IE & Chrome
  4. Burp
  5. Telnet
Source Control:
  1. TFS
  2. TFS online
  3. GIT

Wednesday, 21 February 2018

Consultant Bingo - A master class

I love a useful term to baffle the room as much as the next fellow but watching a master in a meeting today:
STRIDE Model is Microsoft's Security/Threat classification model.  I had to look it up and found another acronym.
DREAD Model is pretty much the same thing.

Give a 'hear-hear' for: "I evaluated my DTAP environments cross Federation services using the STRIDE model over the DREAD model because it is simpler.  Of course all the cross cutting concerns have been dealt with." 

Friday, 9 February 2018

CORS for SharePoint Requests

Problem:  I wish to create a common header for my client to layover multiple applications to tie together branding and global organisation branding.  Similar to what O365 does as shown below:
Provide a common header that logs the user in and dynamically generates the header within SharePoint.  Use an HTTP Javascript request from multiple children applications to deliver the shared user common header.  As I have multiple application on sub-domains (e.g. https://hr.contoso.com https://cash.finance.contoso.com and even https://clients.oldurl.net) so I need to ensure allow CORS requests that also allow for user authentication.  

"The CORS mechanism supports secure cross-domain requests and data transfers between browsers and web servers."  Mozilla

Initial Hypothesis:

Option 1 - IIS and SharePoint struggle to handle this requirement using configuration.  For instance by default, only same origin sub domain requests are allowed.  Adding the header Access-Control-Allow-Origin: * allows for any domain but I can't specify to use credentials so i need an anonymous site and then i loose my ability to identify my user and generate a dynamic menu.
Result: Fail.  I receive the following error in the browser: "A wildcard '*' cannot be used in the 'Access-Control-Allow-Origin' header when the credentials flag is true"

Option 2- Specify a multiple sub-domains i.e. Access-Control-Allow-Origin: https://hr.contoso.com,other.contoso.com
To do authentication I now need the following 3 HTTP response headers:
Access-Control-Allow-Origin: https://hr.contoso.com, https://other.contoso.com
Access-Control-Allow-Credentials: true
Vary: Origin
Result: Fail.  I receive the following error in the browser: "The 'Access-Control-Allow-Origin' header contains multiple values '', but only one is allowed".

Option 3 - Specify a single sub-domains i.e. Access-Control-Allow-Origin: https://hr.contoso.com
Access-Control-Allow-Origin: https://hr.contoso.com
Access-Control-Allow-Credentials: true
Vary: Origin
Result: Fail.  Works for the hr sub-domain but my other sub-domains fail. I have multiple sub-domains that need access.

Key take away: There can only be 1 Access-Control-Allow-Origin response header and the returned Access-Control-Allow-Origin header can only have one URL.

Option 4 - Dynamically inject the Access-Control-Allow-Origin, in SharePoint this is an ISAPI filter or I need to use the Global.asax to dynamically set the HTTP Access-Control-Allow-Origin header to a white-list list of URLs.  Beware of caching pages downstream.  Alternatively, URL Rewrite can be used on the IIS WFE's.

Thanks to Abhieshek Sharma for highlighting my lack of knowledge around CORS requests.

Wednesday, 31 January 2018

Looking for a cheap quick UI testing and monitoring Tool - end test and Ghost Inspector Review

Problem:  My client is looking for a simple tool to monitor a website is up and running and can run a small set of UI tests and asserts to verify it is working as expected.

Initial Hypothesis:  There are a lot of monitoring sites like uptime that meet this requirement but I reviewed Ghost Inspector and endtest.  I am not looking to do full CI as I would look at Selenium WebDriver for an enterprise solution for UI testing.

Resolution:  Trial endtest and Ghost inspector on my O365 subscription to validate it monitors and alerts, can perform advanced logins and it can validate custom pages after JavaScript injection.  Price and feature wise both tools are pretty similar.

Ghost Inspector Initial Thoughts
Easy to use and their is a recording function for Chrome.  This review has put me off Ghost Inspector to some degree but definitely a good product to evaluate.
Bad review for Ghost Inspector but it does assume enterprise level UI testing more suited to tooling like Selenium.

endtest Initial Thoughts
Easy to use, setup testing in a matter of minutes, recorded actions and assertions.  The trial is limited as I could not check the scheduling mechanism but end test looks like the idea tool for my requirement.  Would need to go for the pro licence at $79 per month.  A simpler smaller option would be more attractive but let's see what the client thinks.

Tuesday, 23 January 2018

Basic Branching Strategy for TFS and GIT



  • The main difference between normal TFS branching strategy is that you branch more often for shorter time periods and check in small code change units into the "Development" branch.
  • Delete the black line once the feature is complete and checked back into the Development branch.  Can easily start a new functional local GIT branch to amend the next feature.
Note: Easy to also grab a GIT local branch from the Main branch (inline with you production code base), make changes and then when checked back in they hotfix goes into both the Main and Development code branches.

Friday, 19 January 2018

Interviewing Developers, Leads and Consultants for SharePoint projects

Overview:  Depending on the project will dictate the skills and experience I look for.  This post lists the skills I generally look for when hiring dev and leads for SharePoint based projects.  Firstly, I compile a list of skills for the project and ensure each developer role covers multiple areas/expertise types.  My general list is shown below.

Skill needed:

  • SharePoint
  • CSOM/REST
  • PHA
  • TFS / GIT
  • .NET/C# 
  • WCF / Web API
  • SQL Server 
  • Entity Framework/Code First
  • HTML/CSS
  • JQuery, 
  • Angular JS, KnockOut React VueJS, Other JScript Libraries
  • O365
  • Azure
  • Federation/Security
  • Agile/Scrum

I keep a scorecard and Notes that I fill in for each candidate.  If they score too low in the technical section, I don't start the Personal section, and until I think they are a good candidate then I start the problem solving which I find to be the best indicator of if a guy is going to work out.  Looking back at a lot of developers and leads hired, the 2 critical sections are problem-solving and admits limitations (the guys that don't know when to say "I don't know" are generally a problem if hired). 

Candidate Template:  John Doe

Technical:
Notes
John
SharePoint
Branding, knows SP limits excellent,
8 missed JS injection
SSO/Federation/Security
SAML, ADFS, passive clainms and SSL
9
PHA
Types, S2S vs ACS, Certs, MVC app pkg
8.5
JavaScript/TypeScript/AngularJS/SPFx
Namespaces, versions ng,
9.5
Search
Trimming, CEWS, components, DisplayTemplates, KQL
10
BI
SSRS, Power BI, SSAS, rdl, understand no depth in knowledge
7
                  

Personal:
Notes
John        
Eager

9
Smart
Super adjusted
                              
8
Like

8
Hardworking

9
Admits limitations
                                                               

10




Problem Solving:       
Notes    
Abhishek
Logical

7
SharePoint Problem Solving
                                                               

9
                                   

Smart, nice guy, super knowledgeable.  Admitted he does not know BI at all and then actually gave a solid explanation of BI on SP. 
Technical: 9
Personal: 9

Problem Solving: 8


Example qus when trying to identify a candidates strengths:
QU: Difference/compare Web Services vs WCF vs Web API
ANS:
Web Services is the oldest, .asmx extension are ASP.NET Microsoft's web services.  HTTP protocol only and uses SOAP (XML).  Microsoft proprietary.
WCF was the next release and ends with the extension .svc.  Supports the following protocols: HTTP, HTTPS, TCP, Named Pipes, MSMQ.  WCF uses SOAP (XML)Complex to configure but offers flexibility.  Add REST support using webHttpBindings and then can use XML, JSON and ATOM data format.  IIS needs config change to support PUT and .. verbs.
WebAPI is part of MVC template wasn't originally.  Simple to setup and supports REST.  Lightweight and easy to setup.  Easy to consume.  HTTP protocol only. Supports XML and JSOM data format.


CSS Basic Qus (as I am rubbish, thanks to Jeff H):
QU: How can we add/implemented CSS to our pages (3 approaches)
ANS: Inline css, in the head section of the page or call/reference an external CSS file
QU: Explain Z-Index
ANS: Stack order
QU: Browser engines used
ANS:  IE uses Trident or now called HTMLEdge, Chrome & Opera use Blink; Safari uses webKit; Firefox/Mozilla uses Gecko.
QU: Explain block object positioning between: Absolute, relative and fixed. 
QU: Diff class selector vs and id selector
QU: Explain Display: None vs Display: Hidden








Thursday, 18 January 2018

CSOM TLS Issue - The underlying connection was closed

Problem:  I have a console using CSOM that stopped working when the TLS settings were updated firm-wide.  The communication is between the console and a SharePoint farm, using CSOM, and now it no longer works.  The event log generates the following error message on the client machine: A fatal error occurred while creating an SSL client credential. The internal error state is 10013.

Initial Hypothesis: The outbound HTTPS traffic is the issue as the error is telling me that the error was creating the SSL client credential.  The console runs on a web server and the TLS restriction change has caused the issue.  This issue is that the console running can't create an SSL client credential.  The TLS change was made to the console VM and not the SharePoint farm.


The post below helped me query the windows web servers to check the TLS settings using PowerShell.  I believe the outbound is controlled by the inbound TLS settings.

Resolution:  Change the console to use a know TLS version e.g., TLS1.2 as shown below:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;

Alternatively, revert the TLS setting in the registery.  Obviously, this means your server is more susceptible to attack.

More Info:
https://www.sysadmins.lv/blog-en/test-web-server-ssltls-protocol-support-with-powershell.aspx