Edge side includes macros

Mikulas Dite

I'd like to implement Latte macros for ESI (http://www.w3.org/TR/esi-lang, http caching).

tl;dr: Abstracting from esi, the issue is basically how to render arbitrary parts of a full page and give that a unique url.

Real world usage: overall static homepage with currently logged-in user in header could all be cached but for the name which could be loaded as a separate page, which is obviously much faster than generating the whole page.

The idea is there would be and esi macro, which usage would look like this:

	foo
	{esi snippetName}
		inner content
	{/esi}
	bar

With ESI turned off (development), renders as

	foo
		inner content
	bar

WIth ESI turned on (production), renders as

	foo
	<esi:include="http://nette-generated-url?invalidate=snippetName"/>
	bar

and the generated url would serve whatever was inside the macro (in this case it would be inner content).

Its also possible to render a page usable both with (first part) or without ESI (second part) but that is beyond the point of this question.

	foo
	<esi:remove>
		inner content
	</esi>
	<!--esi
		<esi:include="http://nette-generated-url?invalidate=snippetName"/>
	-->

I was thinking about extending the current snippet system, but unlike the other macros it's a huge hack and I don't really feel extending that would be maintainable.
The thing is, if the esi macros would not be using snippets, whats the best way to generate only the partial template, ie a page that would only return inner content in this example?

Simple solution would be only allowing to ESI include Nette components, but that is not a usable as arbitrary blocks in template.

Has anybody been working with ESI and Nette? How would you approach this?

Btw I've looked at how Symfony solves this and it's fine and all, but not really transferable to Nette…

Last edited by Mikulas Dite (2014-11-11 14:03)

bazo

never worked with esi or heard about it before :) but this has to be doable

why is the invalidation needed?

if you create an esi macro that would store the rendered content in www folder you can easily generate the url to this piece of html
i would base this on cache macro, not snippets

or the code generated for esi needs to be dynamic? am i completely wrong?

Mikulas Dite

It works the other way around. For example, you can have a static page (such as this forum page) that has the same content for all users but for the topmost header that shows logged-in username. There are not expectation for backend caching with esi.

The point is the whole page is cached on http level and only the dynamic part is fetched from backend. So there are two http requests, on for the whole page that somewhere in its content has the <esi:include> tag and one additional request for the snippet itself. Since the additional request can goes through all the servers again, it can either by completely dynamic or be cached on per user basis.

The obvious solution is to create a separate latte with the partial template. The template for the full page would include both the esi:include tag and the esi:remove so it's easier to develop on. And a simple presenter that renders just the partial template without any layout. This works but is hard to maintain.

I suppose the esi macro could act like a block macro in the sense the compiled latte would be callable, but I'm not sure the magic of invoking just the block would work. Also I have no idea how the url should look like: it could be either the same page with an arbitrary arg such as article/1-blabla?renderPartial=foo or it could be a separate url like /esi/foo. The first approach does not work if there is same snippet on many pages (such as page header on many articles). The other approach would require unique snippet name enforcing across all the templates, which is weird.

There is a huge overhead on just invoking php, so while this could easily be solved with just template caching, http caching is way faster (generally like 10 times faster, going from 120ms down to 12ms).

PS: I recommend reading the spec its short and explains it way better than I do.

Edit: It's not obvious but the http cache first either loads the request from cache or from backend and then processes all the esi directives.

Last edited by Mikulas Dite (2014-11-11 13:35)

looky

For the article/1-blabla?renderPartial=foo issue, would it be possible to prefix the esi snippet name with some path to the template, similar to how signal receiver path is generated? That could solve the uniqueness issue, but I'm not sure whether it's feasible..

Mikulas Dite

Right, that could work because the same partial (I'm intentionally not using the word snippet) will always have the same template. I guess that means I could remove the name altogether and just use that. It might be as simple as putting __FILE__ and __LINE__ to the macro, but I have to check if the compiled filename is always the same for same templates between latte cache wipes (because otherwise the http cache could be fetching partial with the original filename and consequently failing to load at all).

Last edited by Mikulas Dite (2014-11-11 14:04)

looky

I don't know how exactly (or even if) you use that, but when I see __FILE__, I see Full Path Disclosure, so careful :)

Mikulas Dite

Yeah, in this ver case that is not a problem since the url is never revealed to the client.

mkoubik

I thought you would use some hash of __FILE__ and __LINE__.