When to use folder in SharePoint list/library?

Just read a nice article. I agree with Joanne Klein, but I only see two practical reasons to use folder.

1. Permission control.
2. Too many items in one list/library.

So, if there is choices, go for metadata!

Office 365 is too complicated!

At the moment, I can see 23 Apps under Office 365.

How can we train thousands of users to understand when to use which one?

In my opinion, we only need three apps, and Dynamics 365 should not be part of Office 365.

1. Synchronous communication: Online chatting service(Teams)
2. Asynchronous communication: Email(Outlook)
3. Information management: SharePoint

Most of those apps should join SharePoint. They are:

Calendar,Delve, Excel, Flow, Forms, OneDrive, OneNote, People, Planner, PowerBI, PowerApps, PowerPoint, Stream, Sway, Tasks, To-Do, Video, Word, Yammer

So, why it is so complicated? For more subscription fees?

https://www.office.com/apps?auth=2&home=1

The way to debug workflow 2013 from SharePoint Online

Thanks for the post from Andrew Connell, we got basic concept of workflow 2013 debugging.

As more and more enterprises migrating their SharePoint to Office 365, we cannot rely on "workflow history list" on debugging.

What's the solution?

So far, the only choice is "replication". Replicating the Online Site collection to On-Premise Dev environment, then test it there through Fiddler.

As Andrew Connell mentioned, we need to build the On-Premise Dev environment carefully, but it's possible to replicate the whole site through third-party migration tool, such as ShareGate, then debug from there.

ShareGate is still expensive (although it's possible the cheapest one comparing to other competent). But it should be all right for medium to large enterprise. It's not such a big number comparing to Office 365 subscription fee of the whole company, anyway.

The confusion when a user just moved from Shared Folder to SharePoint

Traditionally, how a user write a document?

1. Launch a MS Office Program, such as MS Word;
2. Give the document a topic;
3. Put content into it;
4. Save.

The problem with SharePoint is the 4th step: "Save". "How can I save the document to SharePoint?" This is one of the most common questions.

One option is to map a SharePoint document library to local network mapped folder:

For SharePoint On-Premise:

For SharePoint Online:

Then users can save the document to that Shared Drive directly, just like what they did with "Shared Folder".

That works, but then we lost most of the benefit from SharePoint.

"SharePoint" means team work. So if a user wants to write a document, below are the steps.

1. Ask themselves the question: Where should I store this document, so other users can find it easily?
2. Who should have rights to view it, and who should be able to modify it?
3. What kind of metadata should this document has? So users can get the basic information without opening it, such as "due date, document owner, project name, etc.".
4. Go to the SharePoint document library in web browser (IE 11 is recommended at the moment), then click "new" button.

If we want to get thousands of documents to be organised well, please think about the document management (as a team) with each of the documents.

SharePoint cannot do that by itself.

PS: Thanks for the reminding from my colleague Andrew Warland, nowadays, users can save the document to SharePoint sites with the help from the latest MS Office. That saves a lot of trouble.

It's still better to think more for other team members at the very beginning.

Shocking change of Office 365 and AAD licensing

Based on this post , for a company with 1000 users, to prevent users from creating Groups, Microsoft will charge AUD 91700 ( around USD 72000) per year!

So, is this the extra cost of "cloud platform"? How many of this kind of licensing changes are there waiting for us?!

I am speechless now.  :-(


[reference]

Azure Active Directory pricing
https://azure.microsoft.com/en-au/pricing/details/active-directory/

The Price of Office 365 Groups

"I honestly cannot come up with a justification for charging extra for the ability to prevent Groups from being created by every user in your organization."

https://practical365.com/blog/price-office-365-groups/

Some thoughts about Microsoft FLOW

After watching Deep dive: Advanced workflow automation with Microsoft Flow, I have to admit that FLOW is much more powerful than I thought. It can replace SharePoint workflows in most of the cases!

However, as it is designed for power users, I smelled something bad.

1. Now I start to understand that why "everyone needs to learn coding". Simple coding(or drag and drop style software development) allows users to do much more work efficiently.

2. We will get millions of worst "software programmers" to build billions of FLOW modules. These FLOW modules may run very slow, may consume a lot of hardware resource, and almost no one can maintain them. Because these FLOW modules are running in Azure, clients need to pay much higher fee than normal. (Microsoft will be very happy about that, and we cannot blame Microsoft)

3. No user would write document for the FLOW functionalities they build.

4. Fix/improve those FLOW modules is not easy, and troubleshooting on those modules would be nightmare.

5. Who is going to test the FLOW modules built by users? A module may accidentally delete a lot of data (which may not be able of recovering), or send out thousands of emails.

6. For most of the FLOW functionalities, if a developer can do it through C# in one day, he/she may need 3 days to do it through "drag and drop". And it's pretty hard to maintain those functionalities. It would take much more time to make minor changes.

7. Not sure how many security issues it will cause if we allow users to build their own FLOW modules.

8. If Microsoft decides to change/upgrade/obsolete some API/function, who is going to upgrade existing customized FLOW modules?

Conclusion: FLOW is too powerful, so it is not for power users but for developers. Normal power users can use it, but only for very simple functionality, especially when some system other than SharePoint is involved. Only in those cases, FLOW is useful to power users and can improve productivity.

Confusion and complains about Office 365

I am learning Office 365. There are many brilliant ideas in this platform, but also something annoying to me.

Below is my thoughts so far.  It will be keep updated.

[20170704]

1. Microsoft Teams really should be part of "Skype for Business", instead of a separate product.

[update 20170919] Two weeks ago, Microsoft announced that "Skype for Business" is emerged into Microsoft Teams. Great!

2. Documents should not be allowed to be copied from SharePoint Sites to "Teams Files".

It's much better to just create a file link in Microsoft Teams.

3. Outlook Group should be renamed as "Shared Inbox".

The details are described in this post

4. Yammer should be part of SharePoint site, instead of a separate product.

5. All documents should be forced to store in SharePoint Document Library.

In other tools (apps), the documents should be attached at a "file link".

[update, 20170722]

6. Unified API

For example, developers should be able to connect to different APPS by one API command.

[update, 20170726]

7. Add "Fast List" to SharePoint Online

"Fast List" needs to be as fast as SQL table, so all Office 365 items, including conversations, calendar events, etc. can all be stored in SharePoint Online.

[update, 20170731]

8. SPFX solution need "site collection" level deployment scope

"For SharePoint Framework solutions, there is only one deployment scope: tenant wide. Once added, and enabled, in the App Catalog, the SharePoint Framework solution will be available to all Site Collections."

https://github.com/SharePoint/sp-dev-docs/blob/master/docs/spfx/enterprise-guidance.md#sharepoint-framework-deployment-scopes

When should we use "Office 365 Groups"?

[updated regarding "Group Identity", 2017-07-04]

There are quite a lot of posts (such as here, here, and here) trying to explain what "Office 365 Groups" is. But, after reading them, I am still confused.

"Office 365 Groups" covers many areas: Azure AD, permission management, document depository (SharePoint sites), email (Exchange Online), group chat (Microsoft Teams), Yammer, etc. But,

• When should Group be used?
• Should we convert existing Email Distribution List into Group automatically, during the migration to Office 365?
• Should we allow users to create new Group when they believe it is necessary?
• Should we use Group Site to replace normal "Team Site"?
• Should we create a Group for a new project?
• Can one user belong to more than one Group?

After months of investigation, finally, I got my own conclusion. However, it is so faraway from the mainstream opinion, it even shocked myself!

There is only one principle: Office 365 Group should be based on basic "Group Identity Job Role".

Why? Because Group is designed to conquer the challenge of communication around group. Let me explain it through an use two cases here.

1. Communication between a person and a team

A user, let's say, Tom, needs help from DBA team to fix a database issue.

Obviously Tom doesn't care which member of the DBA team can help, and he just want the problem get resolved. The procedure may take months, existing DBA member may takes annual leave or even resign, new DBA may join the team at anytime. The communication between Tom and DBA team should not be affected by the those events, and it happens in many apps, such as SharePoint, Yammer, Outlook, Microsoft Teams, etc.

In other words, Tom wants to talk to DBA team, instead of any specific DBA, without the obstacles of permission management, information sharing, action history tracking.

That's why we need Group. Group should only be created for the bottom level, most basic organisation unit, just above individual user.

We should create a Group for DBA team (if there is more than one DBA in the team, and they all look after all databases), but not for IT department (because there is no such job role called "IT staff").

2. Communication in a team which needs to be shared across the whole team

For small projects, if it needs a lot of communication and those communication need to be shared across the whole team, then, it's better to create a group for it.

So, group members can use different tools to talk to each other, and don't need to worry about permission management or missing relevant project information.

=========

Now, it's easy to answer the previous questions.

• When should it be used?

For each job role, if there are more than one person share the same responsibility, we should create a Group for them.

Normally it's just a few users, but, in some special case, for a role like "help desk", there could be more than 30 people.

• Should we convert existing Email Distribution List into Group automatically, during the migration to Office 365?

No.

Unless the DL is created exactly for a job role.

• Should we allow users to create new Group when they believe it is necessary?

No.

Users should contact Office 365 administrators to create it.

• Should we use Group Site to replace normal "Team Site" during migration (from On-Premise to Office 365)?

No.

Unless the existing team site is created exactly for a job role.

• Should we create a Group for a new project?

It depends.

Normally it's needed for small projects.

• Can one user belong to more than one Group?

Normally, no.

Unless he/she happens to take two roles in two teams.

I know many people have different thoughts about "Office 365 Groups". Any comments are welcome!

PS 1: Microsoft Teams are perfect Application for Office 365 Group. We can create a new team/channel for each topic/task.

PS 2: Microsoft Teams really should be part of "Skype for Business".

PS 3: Documents should not be copied from SharePoint Sites to "Teams Files". It's much better to just create a file link in Microsoft Teams.

My thoughts about "GitLab Outage"

Everyone heard the news of GitLab Outage.

Yes, we should use "tools" to do the administration work, and always test those tools in DEV environment first.

For SharePoint, although I have some tools to help manage site collections, the easier way is to build some "PowerShell" scripts toolsets.

Apart from that, in my opinion, the most important part is to split the huge database into many small databases, and keep those databases light-coupling.

Yes, it's just like what SharePoint does with site collections.

I have to confess: although I tried my best not to touch the production environment, and not to log on as system administrator, but, sometimes, I still deleted something accidentally, in Production environment!

Lucky that there are more than 150 site collections. So the damage only affect a small group of users, and it's not too hard to "recover" the changes.

:-)

Why Microsoft web sites are so slow?

It's always slow, especially the MSDN web sites.  Why?

Below is a screenshot from the document "Office 365 for IT Pros.201611"(Page 73). It explains the passive authentication flow, which is the default user authentication mechanism for web browsers.

In theory, after log in, the user credential/ticket is cached, and the rest operation should be faster. But unfortunately, there are a lot of dynamic data loading on MSDN, which may triggered other authentication events.

Maybe that's the reason.

PS: why Google web sites are much faster?

Best simple guide about SharePoint search

Found an excellent post about SharePoint Search

It covers all major parts, and can be read through in a few minutes.

Strongly recommend it!

All credit goes to icansharepoint

No way to host server side code in container

When "SharePoint Framework" was released, I was pretty sure that, sooner or later, Microsoft would host server side code in containers (http://fangdahai.blogspot.com.au/2016/05/the-future-of-sharepoint-development.html)

After a few days of testing the container feature on Windows Server 2016, I realised that it was never going to happen :-(

The installation media of SQL Server Express 2016 with SP1 is around 421 MB (64 bit). After put it into windows container, how much disk space it takes? 10 GB!

The post Windows Containers Compared: WinDocks vs. Microsoft is correct. There is something wrong with "Windows Container" by design. To me, "Windows Container" is more like "Windows Nano Server" virtual machine.

SharePoint needs lightning fast list and library

After the performance test last time , I realized that SharePoint was still so slow comparing to database system (such as SQL Server). I understand that SharePoint is designed for "people access", not a data repository for other computer system, from the very beginning.

But things are changed.

For current enterprises, SharePoint and commercial database system are the major places to host data. Database is fast, but it's really designed for developers. Normal business users cannot easily create tables, add columns, enter data, and manage the whole database. So, they don't have choice. If there are some data they want to share with other people, they have to put it into SharePoint. What if they want to share 50,000 records? They have to copy them to a SharePoint list. It's painfully slow.

We know why SharePoint is so slow. It provides item-level permission control, versioning, auditing......What if users don't need those features, and just want to share a lot of data with other people or another computer system?

That's what is needed: fast list and library. Users still can create list, add/remove columns, create list view......but no other extra features. It's like a database table in SharePoint, but we still can use CAML query to retrieve data, we still create create list views.

If the list/library is created for people access, it should be based on traditional list templates. If we just need something similar to database table, then we can use "fast list".

"Fast list" is even critical for Office 365: if Microsoft wants to use SharePoint as the major place to host data, performance is extremely important.

By the way, do we still need MS Access then?   :-)

The smart watch everyone needs

There are many different types of smart watch on the market. The question is, do we really need them? Do they really change our life style? How many people need to record their heart beat / blood pressure / location all the time? Do we enjoy playing games on "watch"? Do we really want to use "smart watch" to replace "smart phone"? All current smart watches are developed in wrong direction. Watch is very small, but most vendors want to copy the features from smart phone to it! It's like what Microsoft did before: replicate Windows OS to smart phone. This attempt is doomed to fail from the very beginning. From my point of view, a smart watch everyone needs is a simple one with the features below. No fancy appearance, and it is not going to compete with smart phone. 1. Identity verification. I don't want to carry keys, security passes, membership cards, discount cards, library card, drive license, credit cards, transportation card, cash, etc. And, I even don't need to carry mobile phone everywhere. Smart watch is a perfect device for identity verification: light, handy, not likely to left on car / office / home / hotel, or get stolen. Work with fingerprint scanning, it's also much more secure against hacker's attack. At checkout desk of supermarket, no need to take credit card / cash / smart phone out of pocket any more, just raise your hand, put your finger on the watch, "Beep", done. Do we still need wallet? I doubt it. 2. Receive “Check Code” How many passwords do we need to memorize? How complicated each one need to protect you to resist attacks? With the help of this watch, all online system can turn into "passwordless". Want to log on email system? Please enter your email address, then click "submit" button, wait for a few seconds later, enter the "Check Code" just received on watch, then click "submit" again, done. 3. Receive notification For any important action online, such as payment, no matter it's through credit card or PayPal, or just getting the salary of last week, we can get notification in almost real time.
The same notification can also be sent to specified email account.
So, if anything goes wrong, we can respond in short time. 4. Long lasting battery We don't need to keep the watch running all the time. It should be "wake up" instantly when making payment, and then go to sleep again. So, we may only need to run it for a few minutes a day, which means, it can last for months in one charge! When the battery needs recharge, we will get notification. 5. Remote control / No keyboard. We should not be able to enter any text on the watch. So, all configuration is done through computer or smart phone remotely. That's one of the major differences between Watch and Phone. 6. Tough No dust, no water, no shock, and works well in extreme temperature. 7. Enclosed environment This watch is critical to user's daily life, so, security gets highest priority. To minimize risk, it should not allow any third-party software. 8. Date / Clock Only provide basic information. 9. GPS and Emergency report No map needed, but it's nice to know that you can always get help, anywhere, anytime. 10. Privacy OK, this is the inevitable drawback of this watch. But, as it doesn't collect information more than current mobiles, I assume it's fine. How much I am happy to pay for this kind of watch? $200 is fine. But, I think, Google may be happy to give people for free, as exchange of their personal information.

Office 365 portal is awkward, confusing and daunting

It's really designed by and for developers. :-)

How many users are there happy to spend time to get familiar with these 18 icons, to figure out which one they should click, and then start to work there?

What users want to do here?

1. Create / modify a document (OneDrive, SharePoint, Video, Word, Excel, PowerPoint, OneNote, Sway)
2. Talk / contact some one. (Mail, People, Groups. Where's Lync?)
3. Figure out what need to do. (Mail, Calendar, Planner, Tasks, Delve)
4. Web surfing. (Yammer, Newsfeed, Delve)

If users could only see 4 entry icons here, I believe that they would feel much more comfortable.

But, it's still a bit messy.

SharePoint is the place to store information. Are emails information? How about People contact details? Calendar events? Tasks? Why so much information is stored in other separate places?

If we could store all information in SharePoint, then many of those services could be moved into SharePoint User Profile module, such as Mail, People, Groups, Calendar, Planner, Tasks, Yammer, Newsfeed. (OneDrive and Delve are already there)

We even don't need Microsoft Graph anymore! That's a big relief to developers, as they only need to learn how to access SharePoint data.

Do you like this idea?  Any comments are welcome!

[update, 2016-09-29]

If a document needs to be shared in SPO, Yammer, Planner and Groups, does that mean that the same document has 4 separate copies?

To manage the versions and permissions of this document must be hard.  :-)

SharePoint Online - replace Server-side programming with Client-side programming

Finally, with the help of WebHooks, we can use Client-side programming to replace Server-side programming in SharePoint. ( http://www.paitgroup.com/microsoft-renews-its-vows-with-sharepoint/ )

But, does it really resolve the problem of customization on SharePoint Online sites?

In many cases, YES, but we need to be very careful. Because "data communication" is moved from RAM-RAM to Computer-Computer.

1. Hardware Latency

Latency of communication between different processes on the same machine, is totally different from the one between different machines. Let's check it here ( https://gist.github.com/jboner/2841832 ). "Main memory reference" consumes around 100 ns, and "Round trip within same data center" takes around 500 us, which means the latter is 5000 times slower.

For external servers (not in the same data center), "Round trip" may take more than 30 ms. That's 300,000 times slower.

Caching doesn't help much in many cases.

2. Stability

Let's assume that all servers are in the same data center. Is the network in a data center as robust as the RAM on one computer?

3. Extra hardware overhead

How much work it needs to handle a web service request? Please check here for "IIS Architectures" http://www.iis.net/learn/get-started/introduction-to-iis/introduction-to-iis-architecture

How much extra CPU, Memory Access, DISK IO, Network IO will be consumed for each request? Do we need to pay for that?

A data center may handle one million concurrent users easily, but, when one user open a customized page, it may cause many HTTP(s) requests by the JavaScript on that page. And, each triggered workflow instance may also submit many HTTP(s) requests!

4. Development and trouble shooting

To move a workflow activity from "Server Object model" to "Client object model", for me, it's painful.

SharePoint 2013 CSOM APIs are powerful, but, because there is one more layer, it's more complex. However, this post suggests to utilize mature third-party APIs instead of "reinventing wheels". I totally don't agree about that, because of "Reliability".

5. Reliability

If everything is on-premise, for a normal middle size enterprise, they may utilize 10,000 APIs(through assemblies) from 20 different software vendors. That's fine. Everything is fully tested before deploying to production servers. Any update/patch will also be fully tested on non-production servers.

But, if there are 10,000 APIs(through Web Services) from 20 different vendors, how can we keep the whole systems stable? If, on average, each API is upgraded/changed every 10 years, then there will be 3 APIs(Web Services) being changed every day. And not likely the changes can get fully tested on non-production environment first.

In general, the quality of Microsoft products is pretty good, but, how many times Microsoft recalled updates of their products? Can we expect the software/service/APIs from those 20 vendors are all as good as the one from Microsoft? How much we need to pay for these APIs every year?

In summary, we can move everything into cloud, just need to be cautious.

The future of SharePoint development

SharePoint 2016 is finally released in GA. The astonishing thing is "SharePoint Framework".

It seems Microsoft stopped pushing "Addins (APPS)", and invented another development model. I completely understand why some SharePoint experts not happy about that.

I am OK about it, because I didn't spend much time on "Addins" model, and no plan to study "SharePoint Framework".  :-)

I don't know why Microsoft keep building these complex development model. This is just like how Microsoft Vista handle the security problem: pop up a prompt window, let users decide whether accept the potential threat. If the user choose the wrong one......Microsoft doesn't need to be responsible for that, right?

Same problem with full trust farm solution. There are a lot of poor quality fully trusted code, and Microsoft's solution is simple: ban all of them. If users want the functionality, they have to run it in other application servers. If the other application servers crashed because of the poor quality code......Microsoft doesn't need to be responsible for that, right?

Client-side application is very important. I have no doubt about that. But, use it to replace server-side application? In many cases, that make things very complicated. And complexity make poor quality code much worse! And, it decreases the productivity of development, A LOT! (Think about latency, caching, stability, network bandwidth, etc.)

Kicking out the problem, is not really a solution.

What's the real solution?  I have some ideas here.

Split the development into two categories: server-side and client-side.

For "server-side", put each fully trusted solution into a dedicated "container". If the code crashed, the container will crash and reset, and it doesn't affect the other parts of SharePoint. Many of the SharePoint OOTB service instances should also run in "container".

For "client-side", it's actually JavaScript running in something like Content Editor Web Part, with the help of OOTB toolbox/library. (Users can build TypeScript then turn it into JavaScript, of course)

Can we get that in SharePoint 2019? Let's wait and see.

Any comments are welcome.


PS: What is going to replace InfoPath?!



[update, 2016-05-20]

Don't get me wrong. SharePoint 2016 is good, just not so friendly to developers.  :-)

What is "DevOps" anyway?

When I first time saw the word "DevOps", I thought it's quite straightforward: a system administrator with software development skill. That's why it is called "DevOps", not "OpsDev", isn't it?

No. Based on wikipedia, it's the opposite.

It seems that the "DevOps" concept is created purely for software development improvements.

All right. Then what should we call it as "a system administrator with software development skill"? Don't know. And unfortunately, I am one of that kind of guys.

:-)


( This post is inspired by Managed services killed DevOps )

Let's remove all the grasses and shrubs, and only leave trees there

So, the free SharePoint Foundation Server is removed from SharePoint Server 2016.

That reminds me of an old story: a king hates all the bugs, worms and snakes, so he ordered his soldiers to remove all grasses and shrubs in the forest. "Anyway, we only need wood, right?"

What happened to the forest in the end?!

[update, 2016-02-17]

There are so many free alternatives. If Microsoft doesn't provide the free SharePoint Foundation Server, potential users will choose those platforms. Then, eventually, SharePoint will get less and less developers, administrators and users. So, its competitors will get bigger and bigger market share!

Every politician should learn SharePoint

We all heard the sad news from Mecca. When I see the picture below, I realized that there was something wrong.

Some people said that every one should learn to code, I don't agree about that. But, maybe, every politician should learn SharePoint.

Anyone who is familiar with SharePoint knows that we should not put all data into one site collection. Instead, we split the data into different category, and then create an dedicated and isolated site collection for each of the category.

If we split those 2 million people into 1000 groups, and then set 20 meter gap among those groups, would similar stampede ever happen?

100,000 police deployed there didn't help much. It's like, no matter how many CPU cores, how much RAM and how many SSD are there, for a SharePoint farm with even 2000 users, if we put all data into one site collection, sooner or later, we will see the sad ending.