Category: Sharepoint


Microsoft’s file systems organize storage devices based on cluster size. Also known as the allocation unit size, cluster size represents the smallest amount of disk space that can be allocated to hold a file. Because ReFS and NTFS don’t reference files at a byte granularity, the cluster size is the smallest unit of size that each file system can reference when accessing storage. Both ReFS and NTFS support multiple cluster sizes, as different sized clusters can offer different performance benefits, depending on the deployment.

Full article from MS: https://blogs.technet.microsoft.com/filecab/2017/01/13/cluster-size-recommendations-for-refs-and-ntfs/

Summary:

ReFS cluster sizes

ReFS offers both 4K and 64K clusters. 4K is the default cluster size for ReFS, and we recommend using 4K cluster sizes for most ReFS deployments because it helps reduce costly IO amplification:

  • In general, if the cluster size exceeds the size of the IO, certain workflows can trigger unintended IOs to occur. Consider the following scenarios where a ReFS volume is formatted with 64K clusters:
    • Consider a tiered volume. If a 4K write is made to a range currently in the capacity tier, ReFS must read the entire cluster from the capacity tier into the performance tier before making the write. Because the cluster size is the smallest granularity that the file system can use, ReFS must read the entire cluster, which includes an unmodified 60K region, to be able to complete the 4K write.
    • If a cluster is shared by multiple regions after a block cloning operation occurs, ReFS must copy the entire cluster to maintain isolation between the two regions. So if a 4K write is made to this shared cluster, ReFS must copy the unmodified 60K cluster before making the write.
    • Consider a deployment that enables integrity streams. A sub-cluster granularity write will cause the entire cluster to be re-allocated and re-written, and the new checksum must be computed. This represents additional IO that ReFS must perform before completing the new write, which introduces a larger latency factor to the IO operation.
  • By choosing 4K clusters instead of 64K clusters, one can reduce the number of IOs that occur that are smaller than the cluster size, preventing costly IO amplifications from occurring as frequently.

Additionally, 4K cluster sizes offer greater compatibility with Hyper-V IO granularity, so we strongly recommend using 4K cluster sizes with Hyper-V on ReFS.  64K clusters are applicable when working with large, sequential IO, but otherwise, 4K should be the default cluster size.

NTFS cluster sizes

NTFS offers cluster sizes from 512 to 64K, but in general, we recommend a 4K cluster size on NTFS, as 4K clusters help minimize wasted space when storing small files. We also strongly discourage the usage of cluster sizes smaller than 4K. There are two cases, however, where 64K clusters could be appropriate:

  • 4K clusters limit the maximum volume and file size to be 16TB
    • 64K cluster sizes can offer increased volume and file capacity, which is relevant if you’re are hosting a large deployment on your NTFS volume, such as hosting VHDs or a SQL deployment.
  • NTFS has a fragmentation limit, and larger cluster sizes can help reduce the likelihood of reaching this limit
    • Because NTFS is backward compatible, it must use internal structures that weren’t optimized for modern storage demands. Thus, the metadata in NTFS prevents any file from having more than ~1.5 million extents.
      • One can, however, use the “format /L” option to increase the fragmentation limit to ~6 million. Read more here.
    • 64K cluster deployments are less susceptible to this fragmentation limit, so 64K clusters are a better option if the NTFS fragmentation limit is an issue. (Data deduplication, sparse files, and SQL deployments can cause a high degree of fragmentation.)
      • Unfortunately, NTFS compression only works with 4K clusters, so using 64K clusters isn’t suitable when using NTFS compression. Consider increasing the fragmentation limit instead, as described in the previous bullets.

While a 4K cluster size is the default setting for NTFS, there are many scenarios where 64K cluster sizes make sense, such as: Hyper-V, SQL, deduplication, or when most of the files on a volume are large.

Download sysmon:

NEW: Sysmon 6.02 is available ! : https://technet.microsoft.com/en-us/sysinternals/sysmon  and how to use it:

Installation and usage:

List of web resources concerning Sysmon: https://github.com/MHaggis/sysmon-dfir

Mark russinovitch’s RSA conference: https://onedrive.live.com/view.aspx?resid=D026B4699190F1E6!2843&ithint=file%2cpptx&app=PowerPoint&authkey=!AMvCRTKB_V1J5ow

Sysmon config files explained:

https://github.com/SwiftOnSecurity/sysmon-config

https://github.com/ion-storm/sysmon-config/blob/master/sysmonconfig-export.xml

https://www.bsk-consulting.de/2015/02/04/sysmon-example-config-xml/

View story at Medium.com

Else other install guides:

Sysinternals Sysmon unleashed

http://www.darkoperator.com/blog/2014/8/8/sysinternals-sysmon

 

Detecting APT with Sysmon:

https://www.rsaconference.com/writable/presentations/file_upload/hta-w05-tracking_hackers_on_your_network_with_sysinternals_sysmon.pdf

 

https://www.root9b.com/sites/default/files/whitepapers/R9B_blog_005_whitepaper_01.pdf

Sysmon with Splunk:

http://blogs.splunk.com/2014/11/24/monitoring-network-traffic-with-sysmon-and-splunk/

https://securitylogs.org/tag/sysmon/

Sysmon log analyzer/parsing sysmon event log:

https://github.com/CrowdStrike/Forensics/blob/master/sysmon_parse.cmd

https://digital-forensics.sans.org/blog/2014/08/12/sysmon-in-malware-analysis-lab

https://github.com/JamesHabben/sysmon-queries

http://blog.crowdstrike.com/sysmon-2/

logparser: http://www.microsoft.com/en-us/download/confirmation.aspx?id=24659

logparser GUI: http://lizard-labs.com/log_parser_lizard.aspx

http://blogs.technet.com/b/askds/archive/2008/08/12/event-logging-policy-settings-in-windows-server-2008-and-vista.aspx

Windows XP/2003/2012 and greater support drive mapping back to the client workstation during a Terminal Services (Remote Desktop) session. This means you can copy files from the server to the client and vice versa.

Each volume (removable, fixed or network) available on the client workstation is mapped (A for drive A:, C for drive C:, X for drive X: etc) and the remote Terminal Services session inherits the user’s permission. So if you are logged on to the workstation as user A and you log in to the Terminal Services server as user B, the session will have access to the drives according to A’s permissions.

Drives can also be mapped like a network drive. The client drives are accessible as \\TSCLIENT\C. Note the client workstation’s machine name is not used, it is always referenced with the generic name TSCLIENT.

To display the files on TSCLIENT:

DIR \\TSCLIENT\C

So you can map a drive as follows:

NET USE Y: \\TSCLIENT\C

or simply use the Universal Naming Convention (UNC) syntax:

COPY \\TSCLIENT\C\MYDIR\*.XLS D:\DOCUMENTS\ 

example:

ROBOCOPY \\TSCLIENT\C\MYDIR D:\DOCUMENTS *.XLS /Z /ETA

ROBOCOPY \\TSCLIENT\C\MYDIR D:\DOCUMENTS *.* /MIR /Z /ETA /r:1 /w:1 /Log+:d:\log.txt

 

Note: If you receive an “Attempt to access invalid address” error when using the UNC path \\tsclient\c, then the problem is on the client side.

Likely, the Windows firewall is turned on and blocking file shares, or “File and Printer Sharing For Microsoft Networks” is turned off in the NIC properties, the Server service is disabled, or simple file sharing is enabled on the client

 

Using Powershell:

http://msexchange.me/2014/06/05/monitoring-event-id-thru-powershell/

http://community.spiceworks.com/topic/282720-powershell-event-log-monitor-email-alert-script-central-monitor

https://vijredblog.wordpress.com/2014/03/21/task-scheduler-event-log-trigger-include-event-data-in-mail/

Using SCOM:

http://jimmoldenhauer.blogspot.fr/2013/03/scom-2012-how-to-generate-alerts-from.html

http://scomandplus.blogspot.fr/2013/02/creating-rules-to-monitor-security-logs.html

http://thoughtsonopsmgr.blogspot.fr/2013/11/windows-event-log-monitoring-how-to-get.html

http://opsmgradmin.blogspot.fr/2011/05/scom-monitoring-windows-event-logs.html

 

 

 

 

Troubleshooting slow logons:

http://blogs.technet.com/b/askds/archive/2009/09/23/so-you-have-a-slow-logon-part-1.aspx

http://blogs.technet.com/b/askds/archive/2009/09/24/so-you-have-a-slow-logon-part-2.aspx

Logon process: http://fr.slideshare.net/ControlUp/understanding-troubleshooting-the-windows-logon-process

Tools for troubleshooting:

http://social.technet.microsoft.com/wiki/contents/articles/10128.tools-for-troubleshooting-slow-boots-and-slow-logons-sbsl.aspx

http://social.technet.microsoft.com/wiki/contents/articles/10123.troubleshooting-slow-operating-system-boot-times-and-slow-user-logons-sbsl.aspx

And powershell:

http://blogs.citrix.com/2015/08/05/troubleshooting-slow-logons-via-powershell/

Analyze GPOs load time: http://www.controlup.com/script-library/Analyze-GPO-Extensions-Load-Time/ee682d01-81c4-4495-85a7-4c03c88d7263/

 

How to use Xperf, Xbootmgr, Procmon, WPA?

xperf;xbootmgr;xperfview comes from Windows ADK (Windows performance toolkit sub part). Procmon is a sysinternal tool.

http://superuser.com/questions/594625/how-can-i-analyze-performance-issues-before-during-the-logon-process

http://blogs.technet.com/b/askpfeplat/archive/2012/06/09/slow-boot-slow-logon-sbsl-a-tool-called-xperf-and-links-you-need-to-read.aspx

http://social.technet.microsoft.com/wiki/contents/articles/10128.tools-for-troubleshooting-slow-boots-and-slow-logons-sbsl.aspx

Other interesting articles:

http://blogs.technet.com/b/askpfeplat/archive/2014/10/27/becoming-an-wpa-xpert-part-11-troubleshooting-long-group-policy-processing.aspx

https://www.autoitconsulting.com/site/performance/windows-performance-toolkit-simple-boot-logging/

https://randomascii.wordpress.com/2012/09/04/windows-slowdown-investigated-and-identified/

https://randomascii.wordpress.com/2013/04/20/xperf-basics-recording-a-trace-the-easy-way/

 

Windows Performance Analyzer (wpa.exe) youtube: https://www.youtube.com/watch?v=HGTlc_gWH_c

Xperf data collection tool: https://xperf123.codeplex.com/releases/view/66888

 

For boot tracing:

http://www.msfn.org/board/topic/140247-trace-windows-7-bootshutdownhibernatestandbyresume-issues/

xbootmgr -trace boot -traceFlags BASE+CSWITCH+POWER -resultPath C:\TEMP

with boot phases:
xbootmgr -trace boot -traceflags base+latency+dispatcher -stackwalk profile+cswitch+readythread 
       -notraceflagsinfilename -postbootdelay 120 -resultPath C:\TEMP
 

For shutdown tracing:

xbootmgr -trace shutdown -noPrepReboot -traceFlags BASE+CSWITCH+DRIVERS+POWER -resultPath C:\TEMP

For Standby+Resume:

xbootmgr -trace standby -traceFlags BASE+CSWITCH+DRIVERS+POWER -resultPath C:\TEMP

For Hibernate+Resume:

xbootmgr -trace hibernate -traceFlags BASE+CSWITCH+DRIVERS+POWER -resultPath C:\TEMP

replace C:\TEMP with any temp directory on your machine as necessary to store the output files

Analyses of the boot trace:

Boot_MainPathBoot.png

To start create a summary xml file, run this command (replace the name with the name of your etl file)

xperf /tti -i boot_BASE+CSWITCH+POWER_1.etl -o summary_boot.xml -a boot

Analyses of the shutdown trace:

The shutdown is divided into this 3 parts:

Shutdown_picture.png

To generate an XML summary of shutdown, use the -a shutdown action with Xperf:

xperf /tti -i shutdown_BASE+CSWITCH+DRIVERS+POWER_1.etl -o summary_shutdown.xml -a shutdown

 

 

wusa <update>.msu /quiet /norestart /log

example: wusa d:\hotfixes\Windows8.1-KB29456426.msu /quiet /norestart

You can use the Windows Management Instrumentation Command-line (WMIC) to view the installed updates on your computer:

wmic qfe list

Caption CSName Description FixComments HotFixID InstallDate InstalledBy InstalledOn Name ServicePackInEffect Status

Else If the WMIC output is difficult to read, you can use Systeminfo instead, as follows:

systeminfo | findstr /i /c:”KB29456426″

[18]: KB29456426

How to use WUSA with Powershell?

Get-Item .\* | %{Expand-ZipFile -FilePath $_.FullName -OutputPath d:\hotfixes}

Get-Item d:\hotfixes\* | foreach {WUSA “”$_.FullName /quiet /norestart””;while(get-process wusa){Write-Host “Installing $_.Name”}}

Get-HotFix | Where Description -match hotfix
(Get-HotFix | Where Description -match hotfix).count

Introduction:

Event forwarding (also called SUBSCRIPTIONS) is a mean to send Windows event log entries from source computers to a collector. A same computer can be a collector or a source.

There are two methods available to complete this challenge – collector initiated and source initiated:

Parameter Collector Initiated Source Initiated
Socket direction (for firewall rules) Collector –> Source Collector –> Source
Initiating machine Collector Source
Authentication Type Kerberos Kerberos / Certificates

This technology uses WinRM (HTTP protocol on port TCP 5985 with WinRM 2.0, else TCP 80) . Be careful with the Window firewall and configure it to allow WinRM incoming requests.

WinRM is the ‘server’ component and WinRS is the ‘client’ that can remotely manage the machine with WinRM configured.

Differences you should be aware of:

WinRM 1.1
Vista and Server 2008
Port 80 for HTTP and Port 443 for HTTPS

WinRM 2.0
Windows 7 and Server 2008 R2
Port 5985 for HTTP and Port 5986 for HTTPS

WinRM 1.1 can also be downloaded and installed on pre-R2 2003 and XP from here.

Windows Server 2008 Core:

In order to forward events from a 2008 Server that is not R2, you will need to make a few changes. The first change is the default listening port, it needs to be changed from TCP 80 to TCP 5985. Additionally you may need to start the Windows Event Collector Service.

net start wecsvc
winrm set winrm/config/listener?Address=*+Transport=HTTP @{Port=”5985”}

Basic configuration:

on source computers and collector computer:  winrm quickconfig     and add the collector computer account to the local administrators group

To verify a listener has been created type winrm enumerate winrm/config/listener

WinRM Client Setup

Just to round off this quick introduction to WinRM, to delete a listener use winrm delete winrm/config/listener?address=*+Transport=HTTP

on collector computer: wecutil qc. Add the computer account of the collector computer to the Event Log Readers Group on each of the source computers

on collector computer: create a new subscription from event viewer (follow the wizard)

WinRS: WinRS (Windows Remote Shell) is the client that connects to a WinRM configured machine (as seen in the first part of this post). WinRS is pretty handy, you’ve probably used PSTools or SC for similar things in the past. Here are a few examples of what you do.

Connecting to a remote shell
winrs -r:http://hostnameofclient "cmd"
Stop / Starting remote service
winrs -r:http://hostnameofclient "net start/stop spooler"
Do a Dir on the C drive
winrs -r:http://hostnameofclient "dir c:\"

WinRS

Forwarded Event Logs:

This is configured using ‘subscribers’, which connect to WinRM enabled machines.

To configure these subscribers head over to event viewer, right click on forwarded events and select properties. Select the 2nd tab along subscriptions and press create.

This is where you’ll select the WinRM enabled machine and choose which events you would like forwarded.

Subscriptions

Right click the subscription and select show runtime status.

Error 0x80338126

Now it took me a minute or two to figure this one out. Was it a firewall issue (this gives the same error code), did I miss some configuration steps? Well no, it was something a lot more basic than that. Remember earlier on we were talking about the port changes in WinRM 1.1 to 2.0?

That’s right, I was using server 2008 R2 to set the subscriptions which automatically sets the port to 5985. The client I configured initially was server 2008 so uses version 1.1. If you right click the subscription and click properties -> advanced you’ll be able to see this. I changed this to port 80 and checked the runtime status again.

[DC2.domain.local] – Error – Last retry time: 03/02/2011 20:20:30. Code (0x5): Access is denied. Next retry time: 03/02/2011 20:25:30.”

Head back to the advanced settings and change the user account from machine account to a user with administrative rights. After making these changes the forwarded events started to flow.

Subscriptions Advanced

Additional considerations:

In a workgroup environment, you can follow the same basic procedure described above to configure computers to forward and collect events. However, there are some additional steps and considerations for workgroups:

  • You can only use Normal mode (Pull) subscriptions
  • You must add a Windows Firewall exception for Remote Event Log Management on each source computer.
  • You must add an account with administrator privileges to the Event Log Readers group on each source computer. You must specify this account in the Configure Advanced Subscription Settings dialog when creating a subscription on the collector computer.
  • Type winrm set winrm/config/client @{TrustedHosts="<sources>"} at a command prompt on the collector computer to allow all of the source computers to use NTLM authentication when communicating with WinRM on the collector computer. Run this command only once. Where <sources> appears in the command, substitute a list of the names of all of the participating source computers in the workgroup. Separate the names by commas. Alternatively, you can use wildcards to match the names of all the source computers. For example, if you want to configure a set of source computers, each with a name that begins with “msft”, you could type this command winrm set winrm/config/client @{TrustedHosts="msft*"} on the collector computer. To learn more about this command, type winrm help config.

If you configure a subscription to use the HTTPS protocol by using the HTTPS option in Advanced Subscription Settings , you must also set corresponding Windows Firewall exceptions for port 443. For a subscription that uses Normal (PULL mode) delivery optimization, you must set the exception only on the source computers. For a subscription that uses either Minimize Bandwidth or Minimize Latency (PUSH mode) delivery optimizations, you must set the exception on both the source and collector computers.

If you intend to specify a user account by using the Specific User option in Advanced Subscription Settings when creating the subscription, you must ensure that account is a member of the local Administrators group on each of the source computers in step 4 instead of adding the machine account of the collector computer. Alternatively, you can use the Windows Event Log command-line utility to grant an account access to individual logs. To learn more about this command-line utility, type wevtutil sl -? at a command prompt.

References:

http://blogs.technet.com/b/jepayne/archive/2015/11/24/monitoring-what-matters-windows-event-forwarding-for-everyone-even-if-you-already-have-a-siem.aspx

http://blogs.technet.com/b/jepayne/archive/2015/11/20/what-should-i-know-about-security-the-massive-list-of-links-post.aspx

https://technet.microsoft.com/en-us/library/cc748890.aspx

http://windowsitpro.com/security/q-what-are-some-simple-tips-testing-and-troubleshooting-windows-event-forwarding-and-collec

http://technet.microsoft.com/en-us/library/cc749140.aspx

http://blogs.technet.com/b/askperf/archive/2010/09/24/an-introduction-to-winrm-basics.aspx

http://msdn.microsoft.com/en-us/library/aa384372(v=vs.85).aspx

Video:

Tutorials:

1st: Event forwarding between computers in a Domain

http://tutorial.programming4.us/windows_7/Forwarding-Events-(part-1)—How-to-Configure-Event-Forwarding-in-AD-DS-Domains.aspx

2nd: Event forwarding between computers in workgroup

http://tutorial.programming4.us/windows_7/Forwarding-Events-(part-2)—How-to-Troubleshoot-Event-Forwarding—How-to-Configure-Event-Forwarding-in-Workgroup-Environments.aspx

Additional article talking about Event forwarding too:

http://joshuadlewis.blogspot.fr/2014/10/advanced-threat-detection-with-sysmon_74.html

 

The SharePoint Content database’s UserInfo table stores user specific information about an authenticated user and this information and really acts as a cache which is updated periodically. When a first time user first visits a site this table must be populated. There is a row for each user and for each site collection for which they have visited. For example, if you have 100 site collections and 50 users have visited 50 of the site collections there will be 50*50 or 2,500 approximately rows in the UserInfo table.

But the issue is if the user is updated in AD that the UserInfo Table may not be synchronized.

One problem I found recently is a couple of Disabled AD user whose names are still in the people picker results.

You can use the query “WHERE     (SELECT * FROM UserInfo WHERE tp_Title LIKE ‘firstname lastname%’)”

The thing I tried is go to site collection and “All People”, find that user and removed there.

There are bunch of stsadmin command you can run to make sure the userinfo table is up to date:

1. stsadm -o Sync -listolddatabases 30

2. stsadm -o Sync -deleteolddatabases 30

3. stsadm -o Sync -synctiming M:5

4. stsadm -o Sync -sweeptiming M:5

http://blogs.technet.com/b/askds/archive/2011/09/26/advanced-xml-filtering-in-the-windows-event-viewer.aspx

http://blog.oneboredadmin.com/2013/05/filtering-windows-event-log-using-xpath.html