This is a position statement that frameworks are (in all essential manners) languages. This is intentionally a weaker claim than
ApiIsLanguage. See
ApiIsLanguage for context.
RE: "You don't have to learn new syntax or idioms to use an API. APIs aren't language. They're vocabulary."
I've heard this opinion expressed by many different people, but, in general, I disagree.
When the designer of an API has expectations or requirements on how calls to an API are ordered and otherwise synchronized, then the API is not merely a vocabulary; it is a
FrameWork that embodies various protocols and imposes behavioral requirements on the user... sometimes these behavioral requirements are fairly shallow (create comes before delete, don't delete the same thing twice, query before you expect a response, etc.), and sometimes they are very pervasive (special protocols for accessing shared resources to avoid
DeadLock, communications context for security or compression or
DeltaIsolation, contracts with preconditions and postconditions and invariants, temporal and spatial demands to achieve realtime performance or network quality assurance on embedded systems, and so on), but they always exist.
The degree to which an API can be considered a dedicated language is really the degree to which these behavioral concerns are imposed upon the user of the API. Some of these behavioral concerns are derived of
EssentialDifficulty - a
necessary evil in order to achieve properties of the protocol that are not provided directly by the language. Some of the behavioral concerns are derived of
AccidentalDifficulty - an
unnecessary evil that exists wherever the API is dodging around or accommodating limitations of the language, or leaking implementation details.
Examples of
EssentialDifficulty include the need to construct a shared key (e.g.
DiffieHellman) to participate in secure communications with an unknown remote system, or the need to publish data to a named topic before it can be distributed, or the need to register a service before others can find it. Examples of
AccidentalDifficulty include working around the sequential programming model of a language to add concurrency and other synchronization patterns, taking extra steps to ensure an object won't be accessed in one part of the code before destroying it in another, demands on code composition to support
PrevalenceLayer, requiring the programmers carefully ensure that the call stack leading to a particular API call not be involved in certain mutexes for risk of deadlock, sandboxing of code to eliminate as much ambient authority as possible before running scripts and such provided remotely,
BoilerPlateCode to 'initialize' a library (as opposed to individual uses of a protocol), and so on.
AccidentalDifficulty often requires violating
OnceAndOnlyOnce because you need to implement something 'once' in a manner suitable for ABC and 'once' in an incompatible manner suitable for XYZ and so on. Most complexity we experience today is still
AccidentalDifficulty.
This
AccidentalDifficulty is very problematic. Protocols, services, and contracts are difficult enough to compose based purely on
EssentialDifficulty. When
AccidentalDifficulty born of various
MissingLanguageFeatures is added to the mix, well, there is a reason
FrameworksConsideredHarmful:
PrimitivesAndMeansOfComposition is given one very harsh kick to the nads then left crying and whimpering on the floor. So long as you only need the one framework, you'll do well enough, but the framework as a whole is
SecondClass: not abstract, not composable, full of gotchas and idioms that you and any
MaintenanceProgrammer will need to learn and understand, very little obvious from the API itself, and effectively needing its own
FooOneOhOneInSevenDaysForDummiesInaNutshellSuperBibleUnleashed.
It is in these more extreme cases, where the behavior imposed on the user of the API is pervasive and frustrating, that
ApiIsLanguage is clear and obvious. So what's on the other end of that spectrum? Well, at a guess I would say that the other side is where the API feels like a straightforward extension to the language in which it was written, where each element of the API feels and acts and perhaps even
looks (syntactically) like a simple or primitive extension to the language.
SymmetryOfLanguage isn't (further) violated: the result is just as
FirstClass, abstract, and composable as everything else in the language (which, for some languages, might not be a very impressive claim). Users don't need to know anything about the API that they wouldn't also need to know about other language primitives. Etc.
I suspect that, when dealing with APIs at the
FrameWork and pervasive
DesignPatterns end of the API spectrum, use of
TypefulProgramming,
ObjectCapabilityModel,
RealMacros, and other
MetaProgramming mechanisms to guide proper expression of the API is highly justifiable. I don't have numbers, but I suspect both original developers and
MaintenanceProgrammers will have an easier time learning, reading, writing, and verifying the resulting 'language' after such extension efforts. Further, with the
TypefulProgramming approach, or with
FirstClass RealMacros (as supported in Scheme), the IDE and compiler can get in on the action by detecting errors statically, providing syntax highlighting for the extended language, allowing one to drill down into expanded views of macros when debugging, etc. (
DrScheme provides these mechanisms.)
Still, I do understand the folks who resist such efforts, who want to believe
"You don't have to learn new syntax or idioms to use an API. APIs aren't language, they're vocabulary". It would be nice to believe that, no matter what the problem, it can be solved via an API that tosses a few new vocabulary words at it without any unusual requirements being imposed on the user.
If we ever have a language where 100% of frameworks and protocols and unanticipated
DomainValues and entirely new
DomainModels and ad-hoc
MetaModels with their associated inference engines and
MultiValuedLogics can all be expressed without any
unusual behavioral impositions or gotchas relative to the rest of the language - and we achieve this without
LanguageIdiomClutter and with acceptable
EconomyOfExpression (such that it isn't "usual" to jump through hoops) - then
maybe people will stop writing APIs that impose behavioral requirements on the user.
But it isn't possible. Even if we captured all the
AccidentalDifficulty imposed by existing language designs for all known problems, there would still be both
EssentialDifficulty of known problems and
AccidentalDifficulty of the unknown problems. The theoretical best any
LanguageDesigner with intentions towards
MinimalDesign can do is eliminate individual classes of difficulty (one at a time, in an effort with rising combinatorial complexity) then provide the language with a clear path for upgrade while supporting forward compatibility of existing programs. Useful
ExtensibleProgrammingLanguage mechanisms include: some form of extensible syntax, ability to annotate code (
HotComments) and modify the
PostProcessor to operate over them, support for
TypefulProgramming, a standard
HomoiconicLanguage representation of the AST for reflective operation by said
PostProcessor, low-level
DependencyInjection at the code layer (related to
HyperSpaces and
AspectOrientedProgramming), and requiring a clear statement at the top of each page of code describing the exact version of syntax in use.
Once these
ExtensibleProgrammingLanguage mechanisms are in place, they act as a buffer allowing extensions to the language without upgrade to it, and allowing certain classes of upgrades to the language without breaking existing programs or requiring pervasive changes to existing code (since the same
ExtensibleProgrammingLanguage mechanisms can be used to downgrade a syntax to an earlier version). Further, such systems obviate the need for many classes of language upgrade, especially those related to
DomainModelling. If one wants an
InformLanguage style declaration of
FirstClass InteractiveSceneGraphs for a
VirtualWorldInterconnect, one writes up the appropriate extensions to the set of communications services, adds some new syntax for describing and composing
SceneGraphs and the available interactions, and provides the defaults and assumptions in order to hide semantic noise. (That said, as with all
LanguageDesign work, it isn't "trivial" to provide composability, flexibility, static verification, avoid boilerplate, etc. - there are huge differences between well designed language extensions and poorly designed ones just as there are huge differences between well designed APIs and poorly designed ones.)
ExtensibleProgrammingLanguage support reduces the need to perform 'actual' language upgrades to features that either must be pervasive to be useful (such as
DistributedTransactions, subscription to mutable state, automatic distribution of code,
ObjectCapabilityModel, failover redundancy, system-wide
GarbageCollection, or end-to-end
ProcessAccounting policy via
ExplicitManagementOfImplicitContext in a distributed system, etc.) and various forms of expression that cannot be optimized through composition of other primitives (such as supporting 'sets' with their disordered communication rather than just 'lists').
Actually, a greater problem than upgrading a language with new features is upgrading it with
LiberatingConstraints. I would suggest that any
LanguageDesigner consider those at time of
BigDesignUpFront because there is no way to add them after the language becomes popular (that is a penalty of premature success; related is
AlanKayOnMessaging).
ObjectCapabilityModel, compile-time confinement, removal of
AmbientAuthority, some classes of static
TypeSafety, guarantees of Termination (of the sort sought by
TotalFunctionalProgramming), shunning of mutable state (
KillMutableState) and shunning of names (
NominativeAndStructuralTyping) to support distribution, purity in the lower layers, and much more beside all fall into the whole '
LiberatingConstraint' concept.
Well, it seems I've wandered onto subjects that interest me but aren't entirely topical... so I'll stop there.
The point is that
ApiIsLanguage for almost any API that solves a difficult problem. Arguing that one "needs to learn new syntax" may be true but is nonetheless
not (in general) a point against use of an extensible syntax mechanism... because, for APIs, one (in general) "needs to learn the gotchas, idioms,
DesignPatterns, and protocol, that the
RealMacros can guide you in avoiding or utilizing respectively."
Those who argue,
"oh, this new language feature eliminates the need for macros or extensible syntax,", should bear in mind that the new feature likely just
moves the need for macros a bit further towards the vast language frontier, allowing one to deprecate and transition out some language extensions, and perhaps optimize or improve composition of existing language extensions.