README.zxid
###########
<<author: Sampo Kellom�ki (sampo@iki.fi)>>
<<cvsid: $Id: README.zxid,v 1.19 2006/09/05 05:09:37 sampo Exp $>>
1 Who needs this?
=================
ZXID project has currently (Aug 2006) two outputs
libzxid:: A C library for supporting SAML 2.0, including federated Single Sign-On
zxid:: A C program that implements a SAML Service Provider (SP) as a CGI script
You need this if you are
Web Master:: You want to enable SAML based Single Sign-On (SSO) to your web site. In this
case you would use the zxid SP CGI script directly, only configuring
it slightly.
Perl Developer:: You can use the Net::SAML module to integrate SSO
to your application and web site. Given the direct perl support this is
easier than fully understanding the C interface.
PHP Developer:: We expect to support functionality roughly equivalent
to perl Net::SAML. It does not help that swig.org project does
not officially support php5. Perhaps there is something the php5
community could do to fix this? (*** php4 vs. php5 penetration?)
Web Developer:: You want to integrate SAML based SSO to your web site tool or product
so that your customers can enjoy SSO enabled web sites. In this case
you would study zxid.c for examples and use libzxid.a to implement the
functionality in your own program.
Identity Management hacker:: you need some building blocks: you
will study libzxid and add to it, contributing to the project.
ZXID Project has vastly more ambitious goals. See the ZXID Project chapter
later in this document.
2 Installing
============
If you want to try ZXID out immediately, we recommend compiling the
library and examples and installing one of the examples as a CGI
script in an existing web server. See later chapters for more details.
tar xvzf zxid-0.4.tgz
cd zxid-0.4
# N.B. There is no configure script. The Makefile works for all supported platforms as is.
make
make perlmod # optional
cd Net; make install # optional: install Net::SAML perl module
cp zxid <webroot>/
# configure your web server to recognize zxid a CGI, e.g.
mini_httpd -p 8443 -c zxid -S -E zxid.pem
# Edit your /etc/hosts to contain
127.0.0.1 localhost sp1.zxidcommon.org sp1.zxidsp.org
# Point your browser to
https://sp1.zxidsp.org:8443/zxid?o=E
https://sp1.zxidsp.org:8443/zxid.pl?o=E # Perl version
# Find an IdP to test with and configure it...
2.1 Prerequisites
-----------------
This software depends on following packages:
1. zlib from zlib.net. Generally whatever comes with your distro is sufficient.
2. openssl-0.9.8b or later. See www.openssl.org. Generally openssl libraries
distributed with most Linux distros are sufficient.<<footnote: It is
possible to compile without OpenSSL, e.g. for space constrained embedded
system, but this has serious security implications.>>
3. libcurl from http://curl.haxx.se/. I used version 7.15.5, but probably
whatever ships with your distribution is fine. libcurl is needed
for SOAP bindings and for fetching metadata. It needs to be compiled
to support HTTPS.<<footnote: Compilation without libcurl is possible
with some loss of functionality.>>
4. HTTPS capable web server. For most trivial testing CGI support is needed. We
recommend mini_httpd available from http://www.acme.com/software/mini_httpd/
Following additional packages are needed by developers who wish
to build from scratch, including the code generation (the standard
distribution includes the output of the code generation, so most
people do not need these).
a. gperf from gnu.org (only for build process when generating code)
b. swig from swig.org (only for build process and only if you want scripting interfaces)
c. perl from cpan.org (only for build process and only if you want to generate code from .sg)
d. plaindoc from http://mercnet.pt/plaindoc/pd.html (only for build process, for code
generation from .sg, and for documentation)
e. mini_httpd from http://www.acme.com/software/mini_httpd/ (only for canned tutorial)
Although technically not needed to build zxid, you will need an IdP
to test against. We do not, at the time, supply one so you
will need to find a third party.
2.2 Canned Tutorial: Running ZXID as CGI under mini_httpd
---------------------------------------------------------
While zxid will run easily under Apache httpd (see <<link:apache.html:
receipe>>), for sake of simplicity we first illustrate running it with
mini_httpd(8), a very simple SSL capable web server by Jef Poskanzer.
2.2.1 Getting and installing mini_httpd
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can download the source for mini_httpd from
http://www.acme.com/software/mini_httpd/
You should already have installed OpenSSL, or quite probably OpenSSL
shipped with your distribution. If it is not located at
/usr/local/ssl, the you need to edit the mini_httpd ~Makefile~ to
indicate where it is. At any rate you need to uncomment all lines that
start by SSL_ in the ~Makefile~. Then say
make
Now copy the mini_httpd binary somewhere in your path.
2.2.2 Running mini_httpd
~~~~~~~~~~~~~~~~~~~~~~~~
After building zxid, in zxid directory, run
mini_httpd -p 8443 -c zxid -S -E zxid.pem
where
-p 8443 specifies the port to listen to
-c zxid specifies that URL paths ending in "zxid" are CGI scripts
-S specifies that https is to be used
-E zxid.pem specifies the SSL certificate to use
> N.B. The zxid.pem certificate and private key combo is shipped with zxid
> for demonstration purposes. Obviously everybody who downloads zxid
> has that private key, so there is no real security what-so-ever. For
> production use, you must generate, or aquire, your own private
> key-certificate pair (and keep the private key secret). See Certificates
> chapter for further info.
2.2.3 Accessing ZXID
~~~~~~~~~~~~~~~~~~~~
Edit your /etc/hosts file so that the definition of localhost also
includes sp1.zxidcommon.org and sp1.zxidsp.org domain names, e.g:
127.0.0.1 localhost sp1.zxidcommon.org sp1.zxidsp.org
Point your browser to
https://sp1.zxidsp.org:8443/zxid
or if you do not want the common domain cookie check
https://sp1.zxidsp.org:8443/zxid?o=E
2.2.4 Setting up an IdP
~~~~~~~~~~~~~~~~~~~~~~~
Currently zxid does not ship with an IdP (though the necessary
protocol encoders and decoders are latently available in libzxid,
should anyone wish to make an attempt to hack an IdP together).
For you to test zxid, you will need to aquire an IdP from
somewhere - any vendor whose product is SAML 2.0 certified
will do. One possible source is http://symlabs.com/Products/SFIAM.html
who have a free download.
If you do not want to install an IdP yourself (even for testing),
find someone who already runs one and ask if they would be willing
to load the metadata of your zxid SP.
Once you get your IdP up and running, you need to make sure it accepts
the zxid SP in its Circle of Trust (CoT). This is done by placing
the metadata of the SP in right place in the IdP product configuration.
If your IdP supports automatic CoT management, just turn it on
and chances are you are done.<<footnote: On production IdP you should
understand the trust implications (i.e. no trust) of flipping automatic
CoT management on.>>
If not, you can obtain the zxid SP metadata (which is slightly different
for each install so you can't just copy it from existing install) from
https://sp1.zxidsp.org:8443/zxid?o=B
This URL is the +well known location method+ metadata URL. It is also
the SP +Entity ID+ or Provider ID, should the IdP product ask for
this in its configuration. If the IdP product needs you to
supply the metadata manually as an xml file, just point your
web browser to the above URL and save to file.
zxid SP, by default, has automatic fetching of IdP metadata enabled so
there is no manual configuration step needed, provided that the IdP
supports the well known location method. All SAML 2.0 certified IdP
implementations must support it (but you may still need to enable it
in configuration).
However, you will need the Entity ID (Provider ID) of the IdP. This is
the URL that the IdP uses for well known location method of metadata
sharing. You may need to dig the IdP documentation or GUI for a while
to find it. If you already have the IdP metadata as an xml file, open it
and look for EntityDescriptor/entityID. If you already have the
file, you can also import it manually by running following command
./zxid -import file:///path/to/idp-meta.xml
But the preferred method still is just let the automatic method
do its job.
2.2.5 Your first SSO
~~~~~~~~~~~~~~~~~~~~
1. Start at
https://sp1.zxidsp.org:8443/zxid
or
https://sp1.zxidsp.org:8443/zxid?o=E
If you had common domin cookie already in place, and you
are already logged in the IdP, the SSO may happen
automatically (go to step 3). The automatic experience
will be typical when you use SSO regularily for more
than one web site (i.e. SP).
However, if you get a screen titled "ZXID SP SSO",
you need to paste the IdP's Entity ID to the supplied field
and click "Login". If zxid SP already obtained the metadata for the
IdP, you may also see a button specific for your IdP (and in this
case there is no need to know the Entity ID anymore or paste anything).
2. Next step depends on the IdP product you are using. Usually
a login screen will appear asking for user name and password.
Supply these and login. You will need an account at the IdP.
3. For more slick IdPs, that's all you need to do and you will
land right back at the zxid SP page titled "ZXID SP Management".
> Congratulations, you have made your first SSO!
However, some IdPs will pester you with additional questions
and you will have to jump through their hoops. A typical
question is whether you want to accept a federation. You do.
Sometimes the federation question does not appear automatically
and you need to figure out a way to create a federation
in their user interface and how to get them to send you
back to SP. Sometimes the word used is "account linking"
instead of federation.<<footnote: Vendor products are constantly
improving in this area. From protocol perspective
all the additional gyrations are unnecessary. Be sure
to provide feedback to the vendor so that simpler, easier
to use, products will emerge in future.>>
3 Configuring and Running
=========================
ZXID ships with working demo configuration so you can run it right
away and once you are familiar with the concepts, you can return
to this chapter.
ZXID uses a configuration file in hardwired path<<footnote: As of version 0.2 the
configuration file has not been implemented yet. You configure ZXID at compile
time by editing zxidconf.h>>
/var/zxid/zxid.conf
for figuring out its parameters. If this file is not present, built-in
default configuration is used. The built-in configuration will allow
you to test features of ZXID, but should not be used in production
because it uses default certificates and private keys. Obviously the
demo private key is of public knowledge since it is distributed with the
ZXID package, and as such it provides no privacy protection
what-so-ever. For production use you MUST generate your own
certificate and private key.
Usually configuration of a system involves following tasks
1. Configure web server (see your web server documentation)
a. HTTPS operation and TLS certificate. In the minimum you need
the main site, but you may want to configure the Common Domain
Cookie virtual host as well.
b. Arrange for ZXID to be invoked. This could mean configuring
zxid.x or zxid.pl to be recognized as a CGI script, or it could
mean setting up your ~modperl~ or ~modphp~ system to call
ZXID at the appropriate place.
2. Configure ZXID, including signing certificate and CoT with peer metadata
a. generate or aquire certificate
b. Obtain peer metadata (from their well known location) or enable +Instant CoT+ feature.
3. Configure CoT peers with your metadata. They can download your
metadata from your well known location (which is the URL that is your entity ID). For this
to happen you need to have web server and ZXID up and running.
3.1 Configuration Parameters
----------------------------
3.1.1 zxidroot
~~~~~~~~~~~~~~
The root directory of ZXID configuration files and directories. By default this
is /var/zxid and has following directories and files in it
/var/zxid/
|
+-- zxid.conf Main configuration file
+-- pem/ Our certificates
+-- cot/ Metadata of CoT partners (metadata cache)
+-- ses/ Sessions
`-- log/ Log files, pid files, and the like
3.1.2 pem
~~~~~~~~~
Directory that holds various certificates. The certificates
have hardwired names that are not configurable.
ca.pem:: Certification Authority certificates. These are used for validating any
certificates received from peers (other sites on the CoT). The CA certificates
may also be shipped to the peers to facilitate them validating our signatures.
This is especially relevant if the certificate is issued by multilayer CA
hierarchy where the peer may not have the intermediate CA certificates.
sign-nopw-cert.pem:: The signing certificate AND private key (concatenated in one file).
The private key MUST NOT be encrypted (there will not be any opportunity to supply
decryption password).
enc-nopw-cert.pem:: The encryption certificate AND private key (concatenated in one file).
The private key MUST NOT be encrypted (there will not be any opportunity to supply
decryption password). The signing certificate can be used as the encryption
certificate. If encryption certificate is not specified it will default to
signing certificate.
In addition to the above certificates and private keys, you will need
to configure your web server to use TLS or SSL certificates for the main site
and the Common Domain site. We suggest the following naming
ssl-nopw-cert.pem:: SSL or TLS certificate for main site. In order to avoid browser
warnings, the CN field of this certificate should match the domain name of the
site. The SSL certificate can be same as signing or encryption certificate.
cdc-nopw-cert.pem:: SSL or TLS certificate for Common Domain Cookie introduction site.
In order to avoid browser warnings, the CN field of this certificate should match
the domain name of the site. The SSL certificate can be same as signing or encryption
certificate.
3.1.3 cot
~~~~~~~~~
Directory that holds metadata of the Circle of Trust (CoT)
partners. If +Instant CoT+ is enabled, this directory needs to be
writable at run time.
3.2 Special or embedded compile (reduced functionality)
-------------------------------------------------------
libzxid contains thousands of functions and any given application is
unlikely to use them all. Thus the easiest, safest, no loss of
functionality, way to reduce the footprint is to simply enable
compiler and linker flags that support dead function
elimination.<<footnote: Unfortunately the gnu ld does not support dead
function elimination. You should file this as a bug to them. If they
tell you to put evey one of the 7000 some functions in a separate .c file,
consider the scalability implications of this. Read the comments in
pulverize.pl for a full scoop and an approach.>>
If you need to squeeze zxid into as minimal space as possible,
some functionality tradeoffs are supported. I stress that you
should only attempt these tradeoffs once you are familiar with
zxid and know what you are doing. The canned install instructions
and tutorial walk throughs stop working if you omit
significant functionality.
3.2.1 Compilation without OpenSSL
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Comment out the -DUSE_OPENSSL flag from CFLAGS in Makefile and
recompile.
This will cripple zxid from security perspective because it
will no longer be able to verify or generate digital signatures.
Unless your environment does not need trust and security,
or you understand thoroughly how to provide trust and security
by other means, it is a very bad idea to compile without OpenSSL.
N.B. Compiling, or not, zxid with OpenSSL does not affect
whether your web server will use SSL or TLS. Unless you know
what you are doing, you should be using SSL at web server
layer. Given that SSL is used at web server layer, the savings
you would gain from compiling zxid without OpenSSL may be
neglible if you use dynamic linking.
3.2.2 Compilation without libcurl
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Comment out the -DUSE_CURL flag from CFLAGS in Makefile and
recompile.
Disabling libcurl does not have adverse security implications: you
only loose some functionality and depdending on your situation you may
well be able to live without it.
1. Without libcurl, zxid can not act as a SOAP client. This has a few
consequences
a. Artifact profile for SSO is not supported because it needs SOAP
to resolve the artifact. In most cases a perfectly
viable alternative is to use POST profile for SSO.
b. SOAP profiles for Single Logout and NameID management (aka
defederation) are not supported. You can use the redirect
profiles and get mostly the same functionality.
2. Automatic CoT metadata fetching using well known
location method is not supported without libcurl.
You can fetch the metadata manually, e.g. using web browser,
and place it in /var/zxid/cot directory.
If you want to manually control your Circle of Trust
relationships, you probably want to do this anyway so
loss of automatic functionality is a nonissue.<<footnote:
If you compile with libcurl, but still want to disable
automatic metadata fetching, investigate the ZXID_MD_FETCH
and related configuration options.>>
3. Web Services Client (WSC) functionality is not supported
without libcurl. Effectively this is just another case of
SOAP needed. If you have your own SOAP implementation,
you may, at lesser automation, achieve much of the
same functionality by calling the encoder and decoder
functions manually.
3.2.3 Compiling without zlib (not supported)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
zlib is used mainly in redirect profiles. Since zlib foot print is
small, we have made no supported provision to compile without it. If
you hack something together, let us know.
4 Net::SAML Perl Module
=======================
* perl CGI example: zxid.pl
* using with modperl
After building the main zxid tree, you can
cd Net
perl Makefile.PL
make
make test # Tests are extremely sparse at the moment
make install
This assumes you use the pregenerated Net/SAML_wrap.c and Net/SAML.pm files
that we distribute. If you wish to generate these files from origin,
you need to have SWIG installed and then say in main zxid directory
make perlmod
5 PHP Integration
=================
TBD using SWIG
6 Python Integration
====================
TBD using SWIG
7 Java Native API Integration
=============================
TBD using SWIG
8 Native C API
==============
The generated aspects of the native C API are in c/*-data.h, for example
c/saml2-data.h
Studying this file is very instructive.
8.1 C Data Structures
---------------------
From .sg a header (.h) is generated. This header contains structs that
represent the data of the elements. Each element and attribute
generates its own node. Even trivial nodes like strings have to be
kept this way because the nodes form basis of remembering the ordering
of data. This ordering is needed for exclusive XML canonicalization,
and thus for signature verification.<<footnote: It's unfortunate that
the XML standards do not make this any easier. Without order
maintenance requirement, it would be possible to represent trivial
child elements directly as struct fields. An approach that tried to do
just this is available from CVS tag GEN_LALR (ca. 29.5.2006).>>
Any missing data is represented by NULL pointer.
Any repeating data is kept as a linked list, in reverse order of being
seen in the data stream.<<footnote: Reverse order is just an
optimization - or an artefact of simply adding latest element to the
head of the list. If this bothers you, it's easy enough to reverse the
list afterwards. Linked list is simple and works well for data whose
order does not matter much (we use separate pointer for remembering
the canonicalization order) and where random access is not needed, or
cardinality is low enough so that simple pointer chasing is efficient
enough.>>
Simple elements and all attributes are represented by simple string node
(even if they are booleans or integers).
8.1.1 Handling Namespaces
~~~~~~~~~~~~~~~~~~~~~~~~~
An annoying feature of XML documents is that they have variable
namespace prefixes. The namespace prefix for the unqualified elements
is taken to be the one specified in target() directive of the .sg
input. Name of an element in C code is formed by prefixing the element
by the namespace prefix and an underscore.
Attributes will only have namespace prefix if such was expressly
specified in .sg input. However, pointer and length fields
representing attributes take an "a" middlefix, thus giving a separate
C identifier namespace for the attributes.
When decoding, the actual namespace prefixes are recorded. The wire
order encoder knows to use these recorded prefixes so that accurate
canonicalization for XMLDSIG can be produced. The schema order encoder
always uses the prefixes defined using target() directives in .sg
files. The runtime notion of namespaces is handled by ~ns_tab~ field
of the decoding and encoding context. It is initialized to contain
all namespaces known by virtue of .sg declarations. The runtime
assigned prefixes are held in a linked list hanging from ~n~ (next)
field of ~struct zx_ns_s~. (*** more work needed here)
The code generation creates a file such as c/saml2-ns.c which contains
initialization for the table. The main program should point the ns_tab
field of context as follows:
struct zx_ns_s my_ns_tab[] = {
#include "c/saml2-ns.c"
{ 0,0,0,0,0 }
};
main {
struct zx_ctx* ctx;
...
ctx->ns_tab = my_ns_tab;
}
8.1.2 Handling any and anyAttribute
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Since our aim is to be lax in what we accept, every element can handle
unexpected additional attributes as well as unexpected elements. Thus
whether the schema specifies any or anyAttribute or not, we handle
everything as if they were there. However, when attributes and
elements are received out side of their expected context, they are
simply trated as strings whith string names. This is true even for
those attributes and elements that would be recognizable in their
proper context.
The any extension points, as well as some bookkeeping data
are hidden inside ~ZX_ELEM_EXT~ macro. If you tinker with
this macro, be sure you know what you are doing. If you want
to add your own specific fields to all structs, redefining
~ZX_ELEM_EXT~ may be appropriate, but if you want to add more
fields only to some specific structures, you can define
a macro of form
TPF_EEE_EXT
and put in it whatever fields you want. These fields will be
initialized to zero when the structure is created, but are not touched
in any other way by the generated code. In particular, if some of your
fields are pointers, it will be your responsibility to free them. The
standard free functions will not understand to free them. See the data
structure walking functions, below for one way to accomplish this.
8.1.3 Root data structure
~~~~~~~~~~~~~~~~~~~~~~~~~
The root data structure
struct zx_root_s;
is a special structure that has a field for evey top level
recognizable element.
8.1.4 Per element data structures
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*** TBW
8.1.5 Memory Allocation
~~~~~~~~~~~~~~~~~~~~~~~
After decoding all string data points directly into the input buffer,
i.e. strings are NOT copied. Be sure to not free the input buffer
until you are done processing the data structure. If you need to take
a copy of the strings, you will need to walk the data structure as a
post processing step and do your copies. This can be done using
void TPF_dup_strs_len_NS_EEE(struct zx_dec_ctx* c, struct TPF_NS_EEE_s* x);
The structures are allocated via ZX_ZALLOC() macro, which
by default calls zx_zalloc() function, which in turn
uses system malloc(3). However, you can redefine the
macro to use whatever other allocation scheme you desire.
The generated libraries never free(3) memory. In many programming
patterns, this is actually desireable: for example a CGI program can
count on dying - the process exit(2) will free all the memory.
If you need to free(3) the data structure, you will need to walk it
using
void TPF_free_len_NS_EEE(struct zx_dec_ctx* c, struct TPF_NS_EEE_s* x, int free_strings);
void zx_free_any(struct zx_dec_ctx* c, struct zx_note_s* n, int free_strs);
The zx_free_any() works by having a gigantic switch statement that calls
the appropriate specific free function.
You can deep clone the data structure with
void TPF_deep_clone_NS_EEE(struct zx_dec_ctx* c, struct TPF_NS_EEE_s* x, int dup_strings);
struct zx_note_s* zx_clone_any(struct zx_dec_ctx* c, struct zx_note_s* n, int dup_strs);
The zx_clone_any() works by having a gigantic switch statement that calls
the appropriate specific free function.
8.2 Decoder as Recursive Descent Parser
---------------------------------------
The entry point to the decoder is
struct zx_root_s* zx_dec_root(struct zx_dec_ctx* c, int n);
The decoding context holds pointer to the raw data and must
be initialized prior to calling the decoder. The second
argument specifies how many recognized elements are decoded
before returning. Usually you would specify 1 to consume
one top level element from the stream.
The returned datastructure, ~struct zx_root_s~, contains
one pointer for each type of top level element that can
be recognized. The ~tok~ field of the returned value
identifies the last top level element recognized and can
be used to dispatch to correct request handler:
struct TPF_root_s* x = TPF_dec_root(c, 1);
switch (x->gg.g.tok) {
case TPF_NS_EEE_ELEM: return process_EEE_req(x->NN_EEE);
}
When processing responses, it is generally already known
which type of response you are expecting, so you can simply
check for NULLness of the respective pointer in the returned
data structure.
Internally zx_dec_root() works much the same way: it scans
a beginning of an element from the stream, looks up the token
number corresponding to the element name, and switches on
that, calling element specific decoder functions (see next
section) to do the detailed processing.
8.2.1 Element Decoders
~~~~~~~~~~~~~~~~~~~~~~
For each recognizable element there is a function of form
struct TPF_NS_EEE_s* zx_dec_NS_ELEM(struct zx_dec_ctx* c, int tok);
This function works much the same way as ???
8.2.2 Decoder Extension Points
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The generated code is instrumented with following macros
ZX_ATTR_DEC_EXT(ss):: Extension point called just after decoding known attribute
ZX_XMLNS_DEC_EXT(ss):: Extension point called just after decoding xmlns attribute
ZX_UNKNOWN_ATTR_DEC_EXT(ss):: Extension point called just after decoding unknown attr
ZX_START_DEC_EXT(x):: Extension point called just after decoding element name
and allocating struct, but before decoding any of the attributes.
ZX_END_DEC_EXT(x):: Extension point called just after decoding the entire element.
ZX_START_BODY_DEC_EXT(x):: Extension point called just after decoding element tag, including attributes, but before decoding the body of the element.
ZX_PI_DEC_EXT(pi):: Extension point called just after decoding processing instruction
ZX_COMMENT_DEC_EXT(comment):: Extension point called just after decoding comment
ZX_CONTENT_DEC(ss):: Extension point called just after decoding string content
ZX_UNKNOWN_ELEM_DEC_EXT(elem):: Extension point called just after decoding unknown element
Following macros are available to the extension points
TPF:: Type prefix (as specified by -p during code generation)
EL_NAME:: Namespaceful element name (NS_EEE)
EL_STRUCT:: Name of the struct that describes the element
EL_NS:: Namespace prefix of the element (as seen in input schema)
EL_TAG:: Name of the element without any namespace qualification.
8.3 Exclusive Canonical Encoder
-------------------------------
The encoder receives a C data structure and generates a gigantic
string containing an XML document corresponding to the data structure
and the input schemata. The XML document conforms to the rules of
exclusinve XML canonicalization and hence is useful as input to XMLDSIG.
One encoder is generated for each root node specified at the code
generation. Often these encoders share code for interior nodes.
The encoders allow two pass rendering. You can first use the length
computation method to calculate the amount of storage needed and
then call one of the rendering functions to actually render. Or
if you simply have large enough buffer, you can render directly.
8.3.1 Length computation
~~~~~~~~~~~~~~~~~~~~~~~~
Compute length of an element (and its subelements). The XML attributes
and elements are processed in schema order, although this should
not really matter as length in wire order should be the same.
int TPF_enc_len_NS_EEE(struct zx_enc_ctx* c, struct TPF_NS_EEE_s* x);
8.3.2 Encoding in schema order
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Render element into string. The XML attributes and elements are
processed in schema order. This is what you generally want for
rendering new data structure to a string. The wo pointers are not
used.
char* TPF_enc_so_NS_EEE(struct zx_enc_ctx* c, struct TPF_NS_EEE_s* x, char* p);
8.3.3 Encoding in wire order
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Render element into string. The XML attributes and elements are
processed in wire order by chasing wo pointers. This is what you want
for validating signatures on other people's XML documents.
char* TPF_enc_wo_NS_EEE(struct zx_enc_ctx* c, struct TPF_NS_EEE_s* x, char* p);
8.4 Data Accessor Functions
---------------------------
*** TBW
8.5 Memory Allocation and Free
------------------------------
*** TBW
8.6 Walking the data structure
------------------------------
*** TBW
8.9 Thread Safety
-----------------
All generated libraries are designed to be thread safe, provided
that the underlying libc APIs, such as malloc(3) are thread safe.
8 ZXID Project
==============
Immediate goal: build a SAML 2.0 SP and ID-WSF 1.1 WSC
Goals of ZXID project include
* SOAP 1.1 support
* SAML 2.0 compliance
- SP role (highest short term priority)
- IdP role
* Liberty ID-FF 1.2 support
- SP
- IdP
- SAML 1.1
* Liberty ID-WSF 1.1 support
- Discovery bootstrap
- Discovery WSC
- ID-DAP WSC
- ID-DAP WSP
* Liberty ID-WSF 2.0 support
- Discovery bootstrap
- Discovery WSC
- ID-DAP WSC
- ID-DAP WSP
8.1 Project Layout
------------------
Following directory layout is used by the project. Many of the specified
directories are used by intermediate outputs that are not distributed
in tarball releases, but may or may no be present in CVS checkouts.
zxid-0.xx
|
+-- xsd XML schema descriptions of protocols (not distributed)
+-- sg Schema Grammar (.sg) descriptions of protocols
+-- c C code generated from the Schema Grammar descriptions
+-- tex Temporary files for document generation using PlainDoc (not distributed)
+-- html HTML documentation generated using PlainDoc
+-- review Publicly released announcements and documents (not distributed)
+-- t Test scripts and expected test outputs
`-- tmp Temporary files, such as actual test outputs
8.2 Protocol Encoders and Decoders
----------------------------------
The protocol encoders and decoders are generated automatically from
the schema grammar (.sg) descriptions. This ensures accurate protocol
implementation. While the output is strictly schema driven and correct,
the decoders have some provisions to accept some deviations from
strict spec (e.g. out of order elements are tolerated). However,
one should note that XMLDSIG does not tolerate very much deviation,
thus even if decoder accepts a slightly illfomed message, it is likely
to fail in signature verification.
There are three outputs from generation
1. Data structures describing the data (xx.h)
2. Encoder that linearizes the data structure to wire protocol (xx-enc.c)
3. Decoder that converts wire protocol byte stream to a data structure (xx-dec.c)
9 Code Generation Tools
=======================
Main work horse of code generation is xsd2sg.pl, which serves multiple
purposes
1. Build hashes of all declarations in .sg input. Each hash element consists
of array of elements and attributes, as well as groups and attribute groups.
The type of array element sis determined from prefix, per .sg rules.
2. Expand groups and attribute groups
3. Evaluate each element wrt its type and generate
a. C data structures
b. Decoder grammar
c. Token descriptions for perfect hash and lexical analyzer
d. Encoder C code
The code to build hashes is intervowen in the code that generates .xsd
from .sg. The rest of the generation happens in a function called
generate().
Typical command line (to generate SAML 2.0 protocol engine)
~/plaindoc/xsd2sg.pl -d -gen saml2 -p zx_ \
-r saml:Assertion -r se:Envelope \
-S \
sg/saml-schema-assertion-2.0.sg \
sg/saml-schema-protocol-2.0.sg \
sg/xmldsig-core.sg \
sg/xenc-schema.sg \
sg/soap11.sg \
>/dev/null
<<ignore: ~/plaindoc/xsd2sg.pl -d -gen saml2 -p zx_ -r saml:Assertion -r se:Envelope -S sg/saml-schema-assertion-2.0.sg sg/saml-schema-protocol-2.0.sg sg/xmldsig-core.sg sg/xenc-schema.sg sg/soap11.sg >/dev/null >>
To generate SAML 2.0 Metadata engine you would issue
~/plaindoc/xsd2sg.pl -d -gen saml2md -p zx_ \
-r md:EntityDescriptor -r md:EntitiesDescriptor \
-S \
sg/saml-schema-assertion-2.0.sg \
sg/saml-schema-metadata-2.0.sg \
sg/xmldsig-core.sg \
sg/xenc-schema.sg \
>/dev/null
<<ignore: ~/plaindoc/xsd2sg.pl -d -gen saml2md -p zx_ -r md:EntityDescriptor -r md:EntitiesDescriptor -S sg/saml-schema-assertion-2.0.sg sg/saml-schema-metadata-2.0.sg sg/xmldsig-core.sg sg/xenc-schema.sg >/dev/null >>
9.1 Special Support for Specific Programming Languages
------------------------------------------------------
While C code generation is the main output, and this can always be
converted to other languages using SWIG, sometimes a more natural
language interface can be built by directly generating it.
We plan to enhance the code generation to do something like this. At
least direct hash-of-hashes-of-arrays-of-hashes type datastructure
generation for benefit of some scripting languages is planned.
10 ZXID SP
==========
*** warning: not checked lately, may be wrong!
<<table: ZXID SP URLs
URL Description
============ =======================================================
/zxid Same as o=M. Main convenience entry point
/zxid?o=M SSO with CDC; or management if already logged in
/zxid?o=C Common Domain Cookie (CDC) reader, usually under common domain host name.
/zxid?o=E SSO after CDC read; or management if already logged in.
/zxid?o=P HTTP POST end point. Used for forms and last part of POST profile SSO.
/zxid?o=S SOAP end point (HTTP POST)
/zxid?o=B Get SP metadata (or combined SP and IdP metadata if proxying).
>>
*** add description of CGI fields
10 Certificates
===============
TBD
11 License
==========
Copyright (c) 2006 Sampo Kellom�ki (sampo@iki.fi), All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
12 FAQ
======
*** real user FAQs are still lacking. Maybe this stuff is perfect?
12.8 Author's Pet Peeves
------------------------
1. What is Schema Grammar (.sg) and why are you using it?
* Schema Grammar is a compact formal description of XML documents. It is
mostly bidirectionally convertible to XML Schema (XSD) and captures
the useful essence of most XML schemas.
* Schema Grammars are intuitive and compact, often allowing the
essence to be understood at glance, and even most complex cases
being only about 50% of the volume of the corresponding XSD.
* We use Schema Grammar descriptions because they are more human readable
than XSD and still equally amenable to automated code generation.
* Schema Grammar descriptions are usually converted using xsd2sg.pl, which is
part of the PlainDoc distribution.
* See http://mercnet.pt/plaindoc
* N.B. You do not need xsd2sg.pl or PlainDoc if you just want to compile and use ZXID.
2. What is PlainDoc (.pd)?
* PlainDoc is a document preparation system that uses intuitive plain text files
with minimal markup to generate PDF and HTML outputs.
* We use PlainDoc because it makes it easy to maintain documentation.
* See http://mercnet.pt/plaindoc
* N.B. You do not need PlainDoc if you just want to compile and use ZXID.
3. How come zxid is so heavy to compile?
* SAML 2.0 and related specs have a lot of functionality and detail, even
if you really only need 1% of it. We do not wish to arbitrate which
functionality is best or most needed, so we simply provide it all.
* A lot of the code is generated, thus the input for C compiler is well
in excess of half a million lines of code (of which only about 6k
were written by a human).
* Some of the generated files are gigantic, e.g. Net/SAML/zxid_wrap.c
is over 380k lines. Compiler has to process all of this as a single
compilation unit.
* gcc and gnu ld were, perhaps, not designed to process this large inputs
efficiently. Often the implementation strategy of keeping
everything in memory will cause a smaller machines to swap.
* My 1GHz CPU, 256 MB RAM machine definitely swaps and thus
takes about 45 minutes to compile all this stuff.
* I recommend at least 1GB RAM and 3GHz CPU for development
machine. On such machine, you should be able to build in about 10 min.
4. Why do you not use ./configure and GNU autoconf?
* ~autoconf~ is not for everyone. World does not stop without
~autoconf~. Or indeed need ~autoconf~. It is Yet Another Dependency
I Do Not Need (YADIDNN).
* I find the GNU ~autoconf~ stuff much more difficult to understand than
my own ~Makefile~. Why should I debug ~autoconf~ when I could
spend the time debugging my ~Makefile~ or the actual code?
* I find resolving problems much easier at source code and ~Makefile~ level
than trying to debug a million line script generated by some system
I do not understand (perhaps some hardcore ~autoconf~ advocate could
try to convince me and educate me, but I doubt).
* My policy is to only support systems I have first hand experience with,
or I have trustworthy friends to rely on. It does not help me
to have a system that tries to guess +gazillion irrellevant variables+
to an unpredictable state. It's much easier to stick to standards like
POSIX and make sure you have predictable results from predictable inputs.
* If the deterministic and predictable results are wrong, they can
at least be debugged and fixed with a finite amount of work.
* Supporting all relevant systems manually is not that much of work. The
inhabitants of the orrelevant systems can support themselves, probably
learning a great deal on the side.
12.9 Annoyances and improvement ideas
-------------------------------------
There is a lot of commonality that is not leveraged, especially in the
way service end points are chose given the metadata. The descriptors
are nearly identical so casting them to one should work.
Many of the SAML2 responses are nearly identical. Rather than
construct them fully formally, we could have just one "SAML any
response" function. Perhaps this could be supported by some schema
grammar level aliasing feature: if an element derives from base type
without adding anything at all of its own, we might as well only
generate code for the base type.
<<htmlpreamble: <title>ZXID Home</title><body bgcolor="#330033" text="#ffaaff" link="#ffddff" vlink="#aa44aa" alink="#ffffff"><font face=sans><h1>README ZXID</h1> >>
<<EOF: >>