Sandcastle build is taking hours to complete

Topics: Developer Forum, User Forum
Jul 28, 2008 at 3:59 PM
Edited Jul 28, 2008 at 4:05 PM
Hi,

I am new to using Sandcastle and have followed the instructions in  http://ozgrant.com/2008/02/19/building-sandcastle-documentation-with-team-build/#comment-16374.  The issue is it is taking 45 hours to get one chm file created.  I used the Sandcastle Help Builder GUI and created a template (shown below) .  There are 3 dlls I have added to the ComponentConfigurations section which I believe will make it faster as it will cache the info - is that correct?

Can anyone point me in the right direction about option settings that will get the time down?  There are only about 16 dlls being built from our code but it is pulling in Microsoft EnterpriseLibrary dlls and a couple of sql dlls as well.

Any help would be greatly appreciated.

thanks
Kim.

<project schemaVersion="1.6.0.7">
    <componentConfigurations>
        <component id="Cached Framework Comments Index Data" enabled="True" configuration="&lt;component id=&quot;Cached Framework Comments Index Data&quot; type=&quot;SandcastleBuilder.Components.CachedCopyFromIndexComponent&quot; assembly=&quot;{@SHFBFolder}SandcastleBuilder.Components.dll&quot;&gt;&#xD;&#xA;&lt;index name=&quot;comments&quot; value=&quot;/doc/members/member&quot; key=&quot;@name&quot; cache=&quot;100&quot;&gt;&#xD;&#xA;{@CachedFrameworkCommentList}&#xD;&#xA;{@CommentFileList}&#xD;&#xA;            &lt;/index&gt;&#xD;&#xA;&lt;copy name=&quot;comments&quot; source=&quot;*&quot; target=&quot;/document/comments&quot; /&gt;&#xD;&#xA;&lt;/component&gt;" />
        <component id="Cached MSDN URL References" enabled="True" configuration="&lt;component id=&quot;Cached MSDN URL References&quot; type=&quot;SandcastleBuilder.Components.CachedResolveReferenceLinksComponent&quot; assembly=&quot;{@SHFBFolder}SandcastleBuilder.Components.dll&quot;&gt;&#xD;&#xA;&lt;cache filename=&quot;{@AppDataFolder}Cache\MsdnUrl.cache&quot; /&gt;&#xD;&#xA;&lt;targets base=&quot;{@SandcastlePath}Data\Reflection&quot; recurse=&quot;true&quot; files=&quot;*.xml&quot; type=&quot;{@SDKLinks}&quot; /&gt;&#xD;&#xA;&lt;targets files=&quot;reflection.xml&quot; type=&quot;{@ProjectLinks}&quot; /&gt;&#xD;&#xA;&lt;/component&gt;" />
        <component id="Cached Reflection Index Data" enabled="True" configuration="&lt;component id=&quot;Cached Reflection Index Data&quot; type=&quot;SandcastleBuilder.Components.CachedCopyFromIndexComponent&quot; assembly=&quot;{@SHFBFolder}SandcastleBuilder.Components.dll&quot;&gt;&#xD;&#xA;&lt;index name=&quot;reflection&quot; value=&quot;/reflection/apis/api&quot; key=&quot;@id&quot; cache=&quot;10&quot;&gt;&#xD;&#xA;  &lt;cache base=&quot;{@SandcastlePath}Data\Reflection&quot; recurse=&quot;true&quot; files=&quot;*.xml&quot; cacheFile=&quot;{@AppDataFolder}Cache\Reflection.cache&quot; /&gt;&#xD;&#xA;  &lt;data files=&quot;reflection.xml&quot; /&gt;&#xD;&#xA;&lt;/index&gt;&#xD;&#xA;&lt;copy name=&quot;reflection&quot; source=&quot;*&quot; target=&quot;/document/reference&quot; /&gt;&#xD;&#xA;&lt;/component&gt;" />
    </componentConfigurations>
    <ProjectSummary />
    <MissingTags>Summary, Parameter, Returns, AutoDocumentCtors, Namespace, TypeParameter</MissingTags>
    <VisibleItems>InheritedMembers, InheritedFrameworkMembers, Protected, SealedProtected</VisibleItems>
    <HtmlHelp1xCompilerPath path="C:\Program Files\Sandcastle\HTML Help Workshop\" />
    <HtmlHelp2xCompilerPath path="" />
    <OutputPath>.\Help\</OutputPath>
    <SandcastlePath path="" />
    <WorkingPath path="" />
    <CleanIntermediates>True</CleanIntermediates>
    <KeepLogFile>True</KeepLogFile>
    <BuildLogFile path="" />
    <HelpFileFormat>HtmlHelp1x</HelpFileFormat>
    <CppCommentsFixup>False</CppCommentsFixup>
    <FrameworkVersion>2.0.50727</FrameworkVersion>
    <IndentHtml>False</IndentHtml>
    <Preliminary>False</Preliminary>
    <RootNamespaceContainer>False</RootNamespaceContainer>
    <RootNamespaceTitle />
    <HelpTitle>A Sandcastle Documented Class Library</HelpTitle>
    <HtmlHelpName>Documentation</HtmlHelpName>
    <Language>en-US</Language>
    <CopyrightHref />
    <CopyrightText />
    <FeedbackEMailAddress />
    <FeedbackEMailLinkText />
    <HeaderText />
    <FooterText />
    <ProjectLinkType>Local</ProjectLinkType>
    <SdkLinkType>Msdn</SdkLinkType>
    <SdkLinkTarget>Blank</SdkLinkTarget>
    <PresentationStyle>Prototype</PresentationStyle>
    <NamingMethod>Guid</NamingMethod>
    <SyntaxFilters>Standard</SyntaxFilters>
    <ShowFeedbackControl>False</ShowFeedbackControl>
    <BinaryTOC>True</BinaryTOC>
    <IncludeFavorites>False</IncludeFavorites>
    <CollectionTocStyle>Hierarchical</CollectionTocStyle>
    <IncludeStopWordList>True</IncludeStopWordList>
    <PlugInNamespaces>ms.vsipcc+, ms.vsexpresscc+</PlugInNamespaces>
    <HelpFileVersion>1.0.0.0</HelpFileVersion>
    <ContentPlacement>AboveNamespaces</ContentPlacement>
</project> I have used the
Coordinator
Jul 28, 2008 at 7:04 PM
Edited Jul 28, 2008 at 7:05 PM
Your message appears to have gotten cut off and the configuration you posted lacks info about what you are documenting and the dependencies.  Did you add the dependencies to the Assemblies to Document list or did you add them to the Dependencies property?  Adding them to the Assemblies to Document list will cause them to get documented as well which probably isn't what you want.  Add them to the Dependencies property instead so that it only loads them to resolve references etc.  Only assemblies that you actually want included in the help file should be added to the Assemblies to Document list.

Caching will help up to a point but isn't a cure-all for the long build times especially if you have a large amount of information to document.  Two of the components will cut down on initialization time (less than a minute) and the cached MSDN URL component will help with the URL lookups but on builds that span into hours, it probably won't make much difference.  The machine you build on will also affect the build time.  A slow processor and/or lack of memory can severely impact the build time.  1GB of memory is good but 2GB is better.  Any less than 1GB and it's going to be slow.

Eric
Jul 29, 2008 at 10:44 AM
I found that deleting the contents of the C:\Program Files\Sandcastle\Data\Reflection greatly improved my build times.
As I understand, the files there are used for framework references and in my case this wasn't needed. Still, My build time was cut in minutes not hours..
Coordinator
Jul 29, 2008 at 4:59 PM
That's not the solution.  If you don't need SDK links, set the SdkLinkType property to None.  It will have the same effect with the exception of the 20-30 second startup time while it indexes those items.

Eric
Jul 29, 2008 at 6:03 PM

Thanks for the information.  Given I am following the instructions from the link http://ozgrant.com/2008/02/19/building-sandcastle-documentation-with-team-build/#comment-16374, do I add these properties into the target I am overwriting in the build script?  See below for example or do I put it in via the Sandcastle GUI. 



<
Target Name="GenerateDocumentation">

 

<

PropertyGroup>

 

<

SandcastleBuidlerPath>C:\Program Files\Sandcastle\Sandcastle Help File Builder\SandcastleBuilderConsole.exe</SandcastleBuidlerPath>

 

<

SandcastleBuilderProjectTemplateFile>$(MSBuildExtensionsPath)\template.shfb</SandcastleBuilderProjectTemplateFile>

 

<

SandcastleBuilderProjectFile>$(MSBuildProjectDirectory)\project.shfb</SandcastleBuilderProjectFile>

 

<

SandcastleBuilderArguments>-assembly=&quot;$(OutDir)JP.Core.*.dll&quot;-dependency=&quot;c:\projects\referenced assemblies\*.dll&quot;-outputpath=&quot;$(OutDir).&quot;</SandcastleBuilderArguments>

 

<!--

-dependency=&quot;$(SolutionRoot)\References\*.dll&quot; -->

 

<!--

HACK: For some reason the -outputpath argument requires a '.' on the end, otherwise you get BUILD FAILED: Illegal characters in path. -->

 

</

PropertyGroup>

 

<

Error Text="Sandcastle Help File Builder is not found at $(SandcastleBuidlerPath)." Condition="!Exists('$(SandcastleBuidlerPath)')" />

 

<

Error Text="Sandcastle Help File Builder project template is not found at $(SandcastleBuilderProjectTemplateFile)." Condition="!Exists('$(SandcastleBuilderProjectTemplateFile)')" />

 

<

Copy SourceFiles="$(SandcastleBuilderProjectTemplateFile)" DestinationFiles="$(SandcastleBuilderProjectFile)" />

 

<

XmlUpdate XPath="/project/HelpTitle" XmlFileName="$(SandcastleBuilderProjectFile)" Value="Documentation for $(BuildDefinition)" />

 

<

XmlUpdate XPath="/project/HtmlHelpName" XmlFileName="$(SandcastleBuilderProjectFile)" Value="$(BuildDefinition)" />

 

<

XmlUpdate XPath="/project/FooterText" XmlFileName="$(SandcastleBuilderProjectFile)" Value="Build Number: $(BuildNumber)" />

 

<

Exec Command="&quot;$(SandcastleBuidlerPath)&quot; &quot;$(SandcastleBuilderProjectFile)&quot; $(SandcastleBuilderArguments)" />

 

</

Target>

Sorry if this is basic stuff but I'm finding it problematic to understand how it all pieces together.  And am finding little bits of information out as I go along.  You have been very helpful :)

Cheers
Kim.

 

Coordinator
Jul 29, 2008 at 6:29 PM
Edited Jul 29, 2008 at 6:31 PM
From the look of the SandcastleBuilderArguments option, it looks like you've got it right (-assembly and -dependency).  To turn off the MSDN links, you could add -SdkLinkType=None.  To figure out if it's the build script that's the issue, you may just want to create a project using the GUI that includes the assemblies to document and the dependencies along with any other settings you want to use and just build it to see how long it takes.  You can also run it manually from the command line using "SandcastleBuilderConsole  path_to_project.shfb".

You might also check the build log to see which steps are taking the longest.  Each build step outputs the elapsed time at the end of it.  Most likely it's the BuildAssembler step.

Eric
Jul 30, 2008 at 8:42 AM
Edited Jul 30, 2008 at 8:53 AM
Greetings,

To validate Eric's comment about hardware impacting build times, here's a comparison between two boxes running the same SHFB project against the same assemblies from SandcastleBuilderConsole with the same .NET CLR, Sandcastle and SHFB versions:

My development box:

CPU: AMD Athlon X2 Dual @ 2.42GHz
RAM: 3GB
O/S: 32bit WinXP SP3

BuildReferenceTopics: 2 minutes 48 seconds
Total Build time: 5 minutes 32 seconds



Our build server:

CPU: Intel Xeon E5335 @ 2GHz
RAM: 384MB!
O/S: 32bit Win2003 SP2 running as VMWare image

BuildReferenceTopics: 3 hours 12 minutes 10 seconds
Total Build time: 3 hours 17 minutes 25 seconds


Caveat: I stopped the CruiseControl.NET service on the build server to avoid affecting the test e.g. someone triggering a CI build, but since it's operating in a virtual environment, other virtual images may have interfered with the test by placing load on the host...but there was a reason why I was testing this in the first place!

I guess I shouldn't be complaining about a 3+ hour Sandcastle build when Kim's is taking 45 ;-)

Cheers
Si
Jul 30, 2008 at 9:53 AM
Hi,

Just checked the spec of the box....

Our build server:

CPU: Intel(R) Pentium(R)  4 CPU @ 1.7GHz
RAM: 512MB
O/S: 32bit Win2003 SP2 

So that's definitely not going to help......!!!

Just running through Eric's suggestions now :)

Jul 30, 2008 at 10:39 PM
Eric, My SdkLinkType is already set to none but still that part of the build takes a while. Maybe it's what you call " startup time " but it takes far more than 30 seconds on my machine and it's annoying since even that startup time can be avoided using my (I must agree: Ugly) workaround.
I beleive this deserves an issue.
Jul 31, 2008 at 1:19 AM
I left the same test running overnight on our build server, but this time I enabled AV.

The first test which took 3h:17m had the (Kaspersky) Anti-Virus service disabled, when I re-enabled it for the second test, it took 5h:30m.

Makes sense, and should be easy to fix, but figured it was worth mentioning.
Aug 6, 2008 at 9:20 AM
Hi,

Update on progress.  I followed Eric's suggestion of seeing I could get it built via the GUI but it was still taking hours to build so we changed our build server which now has 2 Gig of RAM in it.  When I ran it via the GUI it took 3mins 28 secs - which is great.

I've still got an issue building the docs with team build.  

(GenerateDocumentation target) ->

c:\projects\JP.Core\BuildType\TFSBuild.proj(247,3): error MSB3073: The command ""C:\Program Files\Sandcastle\Sandcastle Help File Builder\SandcastleBuilderConsole.exe" "c:\projects\JP.Core\BuildType\project.shfb" -assembly="c:\projects\JP.Core\Binaries\Release\*.dll" -dep="c:\projects\Referenced Assemblies\*.dll" -outputpath="c:\projects\JP.Core\Binaries\Release\."" exited with code 2.

I've looked up this up and there is another thread http://www.codeplex.com/SHFB/Thread/View.aspx?ThreadId=23707 that says that return code 2 is a parsing command line options.  So going to take a look at that now :)

Cheers
Kim.

Aug 6, 2008 at 10:14 AM
Hi - I've just run a successful build including generating the documentation and it all ran within 21 mins, alot better than 45 hours :)

Thanks for your help - its nice to know there is good support for this :)